Nov 04 02:05:19.125201 ip-10-0-136-63 systemd[1]: Starting Kubernetes Kubelet...
Nov 04 02:05:19.481107 ip-10-0-136-63 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 02:05:19.481107 ip-10-0-136-63 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 04 02:05:19.481107 ip-10-0-136-63 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 02:05:19.481107 ip-10-0-136-63 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 02:05:19.484733 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484665    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:05:19.484733 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484732    2582 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:05:19.484733 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484736    2582 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:05:19.484825 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484740    2582 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:05:19.484825 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484743    2582 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:05:19.484825 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484746    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:05:19.484825 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484748    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:05:19.484825 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484751    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:05:19.484825 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484754    2582 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:05:19.484825 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484757    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:05:19.484825 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484759    2582 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:05:19.484825 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484762    2582 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:05:19.484825 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484764    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:05:19.484825 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484768    2582 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:05:19.484825 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484771    2582 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:05:19.484825 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484774    2582 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:05:19.484825 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484776    2582 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:05:19.484825 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484779    2582 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:05:19.484825 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484782    2582 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:05:19.484825 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484784    2582 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:05:19.484825 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484787    2582 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:05:19.484825 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484790    2582 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:05:19.484825 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484792    2582 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:05:19.485303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484795    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:05:19.485303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484797    2582 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:05:19.485303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484800    2582 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:05:19.485303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484803    2582 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:05:19.485303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484807    2582 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:05:19.485303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484815    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:05:19.485303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484818    2582 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:05:19.485303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484820    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:05:19.485303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484823    2582 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:05:19.485303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484826    2582 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:05:19.485303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484828    2582 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:05:19.485303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484831    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:05:19.485303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484834    2582 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:05:19.485303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484837    2582 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:05:19.485303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484840    2582 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:05:19.485303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484842    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:05:19.485303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484845    2582 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:05:19.485303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484847    2582 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:05:19.485303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484850    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:05:19.485743 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484852    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:05:19.485743 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484855    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:05:19.485743 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484857    2582 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:05:19.485743 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484860    2582 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:05:19.485743 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484862    2582 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:05:19.485743 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484865    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:05:19.485743 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484867    2582 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:05:19.485743 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484870    2582 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:05:19.485743 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484872    2582 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:05:19.485743 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484875    2582 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:05:19.485743 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484877    2582 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:05:19.485743 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484880    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:05:19.485743 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484882    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:05:19.485743 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484898    2582 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:05:19.485743 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484901    2582 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:05:19.485743 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484904    2582 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:05:19.485743 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484907    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:05:19.485743 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484910    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:05:19.485743 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484913    2582 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:05:19.486197 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484915    2582 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:05:19.486197 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484918    2582 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:05:19.486197 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484920    2582 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:05:19.486197 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484925    2582 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:05:19.486197 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484929    2582 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:05:19.486197 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484933    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:05:19.486197 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484937    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:05:19.486197 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484939    2582 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:05:19.486197 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484942    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:05:19.486197 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484944    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:05:19.486197 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484947    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:05:19.486197 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484950    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:05:19.486197 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484952    2582 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:05:19.486197 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484955    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:05:19.486197 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484957    2582 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:05:19.486197 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484960    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:05:19.486197 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484962    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:05:19.486197 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484965    2582 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:05:19.486197 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484967    2582 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:05:19.486641 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484970    2582 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:05:19.486641 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484973    2582 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:05:19.486641 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484976    2582 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:05:19.486641 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484979    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:05:19.486641 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484981    2582 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:05:19.486641 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484984    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:05:19.486641 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484986    2582 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:05:19.486641 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484989    2582 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:05:19.486641 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484992    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:05:19.486641 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484995    2582 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:05:19.486641 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.484998    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:05:19.486641 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485000    2582 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:05:19.486641 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485003    2582 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:05:19.486641 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485005    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:05:19.486641 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485008    2582 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:05:19.486641 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485034    2582 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:05:19.486641 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485038    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:05:19.486641 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485041    2582 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:05:19.486641 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485044    2582 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:05:19.486641 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485046    2582 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:05:19.487162 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485049    2582 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:05:19.487162 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485051    2582 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:05:19.487162 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485054    2582 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:05:19.487162 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485057    2582 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:05:19.487162 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485059    2582 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:05:19.487162 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485062    2582 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:05:19.487162 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485812    2582 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:05:19.487162 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485819    2582 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:05:19.487162 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485821    2582 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:05:19.487162 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485824    2582 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:05:19.487162 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485827    2582 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:05:19.487162 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485829    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:05:19.487162 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485832    2582 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:05:19.487162 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485835    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:05:19.487162 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485838    2582 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:05:19.487162 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485841    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:05:19.487162 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485843    2582 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:05:19.487162 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485846    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:05:19.487162 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485848    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:05:19.487162 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485851    2582 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:05:19.487656 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485854    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:05:19.487656 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485857    2582 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:05:19.487656 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485860    2582 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:05:19.487656 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485863    2582 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:05:19.487656 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485865    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:05:19.487656 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485868    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:05:19.487656 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485870    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:05:19.487656 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485873    2582 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:05:19.487656 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485876    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:05:19.487656 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485878    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:05:19.487656 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485881    2582 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:05:19.487656 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485899    2582 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:05:19.487656 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485903    2582 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:05:19.487656 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485905    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:05:19.487656 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485908    2582 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:05:19.487656 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485911    2582 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:05:19.487656 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485915    2582 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:05:19.487656 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485919    2582 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:05:19.487656 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485922    2582 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:05:19.488127 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485925    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:05:19.488127 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485927    2582 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:05:19.488127 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485930    2582 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:05:19.488127 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485932    2582 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:05:19.488127 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485935    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:05:19.488127 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485937    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:05:19.488127 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485940    2582 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:05:19.488127 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485943    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:05:19.488127 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485945    2582 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:05:19.488127 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485948    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:05:19.488127 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485951    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:05:19.488127 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485953    2582 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:05:19.488127 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485956    2582 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:05:19.488127 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485960    2582 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:05:19.488127 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485962    2582 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:05:19.488127 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485965    2582 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:05:19.488127 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485968    2582 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:05:19.488127 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485970    2582 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:05:19.488127 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485973    2582 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:05:19.488577 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485976    2582 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:05:19.488577 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485978    2582 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:05:19.488577 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485981    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:05:19.488577 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485983    2582 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:05:19.488577 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485986    2582 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:05:19.488577 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485988    2582 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:05:19.488577 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485991    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:05:19.488577 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485994    2582 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:05:19.488577 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485996    2582 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:05:19.488577 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.485999    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:05:19.488577 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486001    2582 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:05:19.488577 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486004    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:05:19.488577 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486007    2582 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:05:19.488577 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486010    2582 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:05:19.488577 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486012    2582 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:05:19.488577 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486015    2582 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:05:19.488577 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486017    2582 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:05:19.488577 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486020    2582 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:05:19.488577 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486022    2582 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:05:19.488577 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486025    2582 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:05:19.489076 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486028    2582 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:05:19.489076 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486030    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:05:19.489076 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486033    2582 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:05:19.489076 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486036    2582 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:05:19.489076 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486039    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:05:19.489076 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486041    2582 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:05:19.489076 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486045    2582 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:05:19.489076 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486048    2582 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:05:19.489076 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486050    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:05:19.489076 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486053    2582 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:05:19.489076 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486055    2582 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:05:19.489076 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486059    2582 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:05:19.489076 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486063    2582 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:05:19.489076 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486067    2582 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:05:19.489076 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486070    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:05:19.489076 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486073    2582 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:05:19.489076 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486075    2582 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:05:19.489076 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486078    2582 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:05:19.489076 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486080    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:05:19.489076 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486083    2582 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:05:19.489559 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486085    2582 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:05:19.489559 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486088    2582 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:05:19.489559 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486090    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:05:19.489559 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486093    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:05:19.489559 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486096    2582 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:05:19.489559 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486098    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:05:19.489559 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486101    2582 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:05:19.489559 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486103    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:05:19.489559 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486106    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:05:19.489559 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486108    2582 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:05:19.489559 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486111    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:05:19.489559 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486113    2582 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:05:19.489559 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486116    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:05:19.489559 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486119    2582 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:05:19.489559 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486612    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:05:19.489559 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486618    2582 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:05:19.489559 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486621    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:05:19.489559 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486623    2582 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:05:19.489559 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486627    2582 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:05:19.490024 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486629    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:05:19.490024 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486632    2582 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:05:19.490024 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486635    2582 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:05:19.490024 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486648    2582 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:05:19.490024 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486653    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:05:19.490024 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486656    2582 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:05:19.490024 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486659    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:05:19.490024 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486662    2582 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:05:19.490024 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486664    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:05:19.490024 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486668    2582 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:05:19.490024 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486672    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:05:19.490024 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486675    2582 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:05:19.490024 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486678    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:05:19.490024 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486680    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:05:19.490024 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486683    2582 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:05:19.490024 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486685    2582 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:05:19.490024 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486688    2582 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:05:19.490024 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486691    2582 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:05:19.490024 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486693    2582 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:05:19.490476 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486696    2582 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:05:19.490476 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486698    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:05:19.490476 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486701    2582 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:05:19.490476 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486704    2582 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:05:19.490476 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486706    2582 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:05:19.490476 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486708    2582 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:05:19.490476 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486711    2582 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:05:19.490476 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486713    2582 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:05:19.490476 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486716    2582 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:05:19.490476 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486719    2582 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:05:19.490476 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486721    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:05:19.490476 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486724    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:05:19.490476 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486727    2582 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:05:19.490476 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486730    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:05:19.490476 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486733    2582 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:05:19.490476 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486736    2582 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:05:19.490476 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486738    2582 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:05:19.490476 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486741    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:05:19.490476 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486743    2582 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:05:19.490476 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486746    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:05:19.490950 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486748    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:05:19.490950 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486751    2582 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:05:19.490950 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486753    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:05:19.490950 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486756    2582 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:05:19.490950 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486758    2582 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:05:19.490950 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486761    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:05:19.490950 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486763    2582 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:05:19.490950 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486765    2582 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:05:19.490950 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486768    2582 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:05:19.490950 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486770    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:05:19.490950 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486773    2582 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:05:19.490950 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486775    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:05:19.490950 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486778    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:05:19.490950 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486780    2582 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:05:19.490950 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486783    2582 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:05:19.490950 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486785    2582 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:05:19.490950 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486788    2582 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:05:19.490950 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486791    2582 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:05:19.490950 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486793    2582 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:05:19.490950 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486796    2582 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:05:19.491427 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486798    2582 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:05:19.491427 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486801    2582 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:05:19.491427 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486804    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:05:19.491427 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486806    2582 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:05:19.491427 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486810    2582 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:05:19.491427 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486813    2582 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:05:19.491427 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486815    2582 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:05:19.491427 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486818    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:05:19.491427 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486820    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:05:19.491427 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486823    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:05:19.491427 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486825    2582 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:05:19.491427 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486828    2582 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:05:19.491427 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486830    2582 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:05:19.491427 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486833    2582 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:05:19.491427 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486835    2582 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:05:19.491427 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486838    2582 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:05:19.491427 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486840    2582 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:05:19.491427 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486842    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:05:19.491427 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486845    2582 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:05:19.491427 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486847    2582 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:05:19.491939 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486850    2582 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:05:19.491939 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486852    2582 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:05:19.491939 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486855    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:05:19.491939 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486857    2582 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:05:19.491939 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486860    2582 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:05:19.491939 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486862    2582 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:05:19.491939 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486864    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:05:19.491939 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486867    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:05:19.491939 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486870    2582 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:05:19.491939 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486872    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:05:19.491939 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486876    2582 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:05:19.491939 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486881    2582 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:05:19.491939 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486899    2582 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:05:19.491939 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486902    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:05:19.491939 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486905    2582 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:05:19.491939 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486908    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:05:19.491939 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486911    2582 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:05:19.491939 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486914    2582 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:05:19.491939 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486916    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:05:19.492948 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486919    2582 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:05:19.492948 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486921    2582 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:05:19.492948 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.486924    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:05:19.492948 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488490    2582 flags.go:64] FLAG: --address="0.0.0.0"
Nov 04 02:05:19.492948 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488500    2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 04 02:05:19.492948 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488506    2582 flags.go:64] FLAG: --anonymous-auth="true"
Nov 04 02:05:19.492948 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488510    2582 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 04 02:05:19.492948 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488515    2582 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 04 02:05:19.492948 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488518    2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 04 02:05:19.492948 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488522    2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 04 02:05:19.492948 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488526    2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 04 02:05:19.492948 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488530    2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 04 02:05:19.492948 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488533    2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 04 02:05:19.492948 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488536    2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 04 02:05:19.492948 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488539    2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 04 02:05:19.492948 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488543    2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 04 02:05:19.492948 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488546    2582 flags.go:64] FLAG: --cgroup-root=""
Nov 04 02:05:19.492948 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488550    2582 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 04 02:05:19.492948 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488553    2582 flags.go:64] FLAG: --client-ca-file=""
Nov 04 02:05:19.492948 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488556    2582 flags.go:64] FLAG: --cloud-provider="external"
Nov 04 02:05:19.492948 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488560    2582 flags.go:64] FLAG: --cluster-dns="[]"
Nov 04 02:05:19.492948 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488565    2582 flags.go:64] FLAG: --cluster-domain=""
Nov 04 02:05:19.492948 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488568    2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 04 02:05:19.493484 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488571    2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 04 02:05:19.493484 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488574    2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 04 02:05:19.493484 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488577    2582 flags.go:64] FLAG: --container-log-max-files="5"
Nov 04 02:05:19.493484 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488581    2582 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 04 02:05:19.493484 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488584    2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 04 02:05:19.493484 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488587    2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 04 02:05:19.493484 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488591    2582 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 04 02:05:19.493484 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488594    2582 flags.go:64] FLAG: --contention-profiling="false"
Nov 04 02:05:19.493484 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488597    2582 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 04 02:05:19.493484 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488600    2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 04 02:05:19.493484 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488603    2582 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 04 02:05:19.493484 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488606    2582 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 04 02:05:19.493484 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488610    2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 04 02:05:19.493484 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488613    2582 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 04 02:05:19.493484 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488616    2582 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 04 02:05:19.493484 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488619    2582 flags.go:64] FLAG: --enable-load-reader="false"
Nov 04 02:05:19.493484 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488622    2582 flags.go:64] FLAG: --enable-server="true"
Nov 04 02:05:19.493484 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488625    2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 04 02:05:19.493484 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488629    2582 flags.go:64] FLAG: --event-burst="100"
Nov 04 02:05:19.493484 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488632    2582 flags.go:64] FLAG: --event-qps="50"
Nov 04 02:05:19.493484 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488634    2582 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 04 02:05:19.493484 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488638    2582 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 04 02:05:19.493484 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488640    2582 flags.go:64] FLAG: --eviction-hard=""
Nov 04 02:05:19.493484 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488644    2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 04 02:05:19.494107 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488647    2582 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 04 02:05:19.494107 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488650    2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 04 02:05:19.494107 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488653    2582 flags.go:64] FLAG: --eviction-soft=""
Nov 04 02:05:19.494107 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488656    2582 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 04 02:05:19.494107 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488659    2582 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 04 02:05:19.494107 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488662    2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 04 02:05:19.494107 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488666    2582 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 04 02:05:19.494107 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488669    2582 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 04 02:05:19.494107 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488672    2582 flags.go:64] FLAG: --fail-swap-on="true"
Nov 04 02:05:19.494107 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488674    2582 flags.go:64] FLAG: --feature-gates=""
Nov 04 02:05:19.494107 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488678    2582 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 04 02:05:19.494107 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488681    2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 04 02:05:19.494107 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488685    2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 04 02:05:19.494107 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488688    2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 04 02:05:19.494107 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488691    2582 flags.go:64] FLAG: --healthz-port="10248"
Nov 04 02:05:19.494107 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488694    2582 flags.go:64] FLAG: --help="false"
Nov 04 02:05:19.494107 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488697    2582 flags.go:64] FLAG: --hostname-override="ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:05:19.494107 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488700    2582 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 04 02:05:19.494107 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488703    2582 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 04 02:05:19.494107 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488706    2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 04 02:05:19.494107 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488709    2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 04 02:05:19.494107 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488713    2582 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 04 02:05:19.494107 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488716    2582 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 04 02:05:19.494652 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488718    2582 flags.go:64] FLAG: --image-service-endpoint=""
Nov 04 02:05:19.494652 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488721    2582 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 04 02:05:19.494652 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488724    2582 flags.go:64] FLAG: --kube-api-burst="100"
Nov 04 02:05:19.494652 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488727    2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 04 02:05:19.494652 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488730    2582 flags.go:64] FLAG: --kube-api-qps="50"
Nov 04 02:05:19.494652 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488732    2582 flags.go:64] FLAG: --kube-reserved=""
Nov 04 02:05:19.494652 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488736    2582 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 04 02:05:19.494652 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488738    2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 04 02:05:19.494652 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488741    2582 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 04 02:05:19.494652 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488744    2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 04 02:05:19.494652 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488747    2582 flags.go:64] FLAG: --lock-file=""
Nov 04 02:05:19.494652 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488750    2582 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 04 02:05:19.494652 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488753    2582 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 04 02:05:19.494652 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488756    2582 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 04 02:05:19.494652 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488762    2582 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 04 02:05:19.494652 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488765    2582 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 04 02:05:19.494652 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488768    2582 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 04 02:05:19.494652 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488771    2582 flags.go:64] FLAG: --logging-format="text"
Nov 04 02:05:19.494652 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488774    2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 04 02:05:19.494652 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488777    2582 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 04 02:05:19.494652 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488780    2582 flags.go:64] FLAG: --manifest-url=""
Nov 04 02:05:19.494652 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488783    2582 flags.go:64] FLAG: --manifest-url-header=""
Nov 04 02:05:19.494652 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488787    2582 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 04 02:05:19.494652 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488790    2582 flags.go:64] FLAG: --max-open-files="1000000"
Nov 04 02:05:19.494652 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488794    2582 flags.go:64] FLAG: --max-pods="110"
Nov 04 02:05:19.495253 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488797    2582 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 04 02:05:19.495253 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488800    2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 04 02:05:19.495253 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488803    2582 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 04 02:05:19.495253 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488806    2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 04 02:05:19.495253 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488809    2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 04 02:05:19.495253 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488812    2582 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 04 02:05:19.495253 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488815    2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 04 02:05:19.495253 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488822    2582 flags.go:64] FLAG: --node-status-max-images="50"
Nov 04 02:05:19.495253 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488825    2582 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 04 02:05:19.495253 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488827    2582 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 04 02:05:19.495253 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488830    2582 flags.go:64] FLAG: --pod-cidr=""
Nov 04 02:05:19.495253 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488833    2582 flags.go:64] FLAG: --pod-manifest-path=""
Nov 04 02:05:19.495253 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488836    2582 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 04 02:05:19.495253 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488839    2582 flags.go:64] FLAG: --pods-per-core="0"
Nov 04 02:05:19.495253 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488842    2582 flags.go:64] FLAG: --port="10250"
Nov 04 02:05:19.495253 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488845    2582 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 04 02:05:19.495253 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488848    2582 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-066e77ac155495e22"
Nov 04 02:05:19.495253 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488852    2582 flags.go:64] FLAG: --qos-reserved=""
Nov 04 02:05:19.495253 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488855    2582 flags.go:64] FLAG: --read-only-port="10255"
Nov 04 02:05:19.495253 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488858    2582 flags.go:64] FLAG: --register-node="true"
Nov 04 02:05:19.495253 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488861    2582 flags.go:64] FLAG: --register-with-taints=""
Nov 04 02:05:19.495253 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488865    2582 flags.go:64] FLAG: --registry-burst="10"
Nov 04 02:05:19.495253 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488868    2582 flags.go:64] FLAG: --registry-qps="5"
Nov 04 02:05:19.495253 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488871    2582 flags.go:64] FLAG: --reserved-cpus=""
Nov 04 02:05:19.495253 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488874    2582 flags.go:64] FLAG: --reserved-memory=""
Nov 04 02:05:19.495253 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488877    2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 04 02:05:19.495920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488881    2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 04 02:05:19.495920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488893    2582 flags.go:64] FLAG: --rotate-certificates="false"
Nov 04 02:05:19.495920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488896    2582 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 04 02:05:19.495920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488899    2582 flags.go:64] FLAG: --runonce="false"
Nov 04 02:05:19.495920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488902    2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 04 02:05:19.495920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488905    2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 04 02:05:19.495920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488908    2582 flags.go:64] FLAG: --seccomp-default="false"
Nov 04 02:05:19.495920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488912    2582 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 04 02:05:19.495920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488914    2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 04 02:05:19.495920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488917    2582 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 04 02:05:19.495920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488920    2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 04 02:05:19.495920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488924    2582 flags.go:64] FLAG: --storage-driver-password="root"
Nov 04 02:05:19.495920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488927    2582 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 04 02:05:19.495920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488929    2582 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 04 02:05:19.495920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488932    2582 flags.go:64] FLAG: --storage-driver-user="root"
Nov 04 02:05:19.495920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488935    2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 04 02:05:19.495920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488938    2582 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 04 02:05:19.495920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488941    2582 flags.go:64] FLAG: --system-cgroups=""
Nov 04 02:05:19.495920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488943    2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 04 02:05:19.495920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488951    2582 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 04 02:05:19.495920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488954    2582 flags.go:64] FLAG: --tls-cert-file=""
Nov 04 02:05:19.495920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488957    2582 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 04 02:05:19.495920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488961    2582 flags.go:64] FLAG: --tls-min-version=""
Nov 04 02:05:19.495920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488966    2582 flags.go:64] FLAG: --tls-private-key-file=""
Nov 04 02:05:19.495920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488969    2582 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 04 02:05:19.496533 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488972    2582 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 04 02:05:19.496533 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488974    2582 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 04 02:05:19.496533 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488977    2582 flags.go:64] FLAG: --v="2"
Nov 04 02:05:19.496533 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488982    2582 flags.go:64] FLAG: --version="false"
Nov 04 02:05:19.496533 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488987    2582 flags.go:64] FLAG: --vmodule=""
Nov 04 02:05:19.496533 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488991    2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 04 02:05:19.496533 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.488994    2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 04 02:05:19.496533 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489128    2582 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:05:19.496533 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489131    2582 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:05:19.496533 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489134    2582 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:05:19.496533 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489137    2582 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:05:19.496533 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489140    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:05:19.496533 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489142    2582 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:05:19.496533 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489145    2582 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:05:19.496533 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489148    2582 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:05:19.496533 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489153    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:05:19.496533 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489156    2582 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:05:19.496533 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489159    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:05:19.496533 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489162    2582 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:05:19.496533 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489164    2582 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:05:19.496533 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489167    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:05:19.496533 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489169    2582 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:05:19.497117 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489172    2582 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:05:19.497117 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489175    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:05:19.497117 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489177    2582 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:05:19.497117 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489180    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:05:19.497117 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489184    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:05:19.497117 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489186    2582 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:05:19.497117 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489191    2582 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:05:19.497117 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489195    2582 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:05:19.497117 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489200    2582 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:05:19.497117 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489204    2582 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:05:19.497117 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489206    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:05:19.497117 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489209    2582 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:05:19.497117 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489212    2582 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:05:19.497117 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489215    2582 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:05:19.497117 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489218    2582 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:05:19.497117 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489220    2582 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:05:19.497117 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489223    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:05:19.497117 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489226    2582 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:05:19.497552 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489228    2582 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:05:19.497552 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489231    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:05:19.497552 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489234    2582 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:05:19.497552 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489236    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:05:19.497552 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489239    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:05:19.497552 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489241    2582 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:05:19.497552 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489244    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:05:19.497552 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489247    2582 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:05:19.497552 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489250    2582 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:05:19.497552 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489252    2582 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:05:19.497552 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489255    2582 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:05:19.497552 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489258    2582 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:05:19.497552 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489260    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:05:19.497552 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489263    2582 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:05:19.497552 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489266    2582 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:05:19.497552 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489268    2582 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:05:19.497552 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489271    2582 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:05:19.497552 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489273    2582 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:05:19.497552 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489278    2582 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:05:19.497552 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489280    2582 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:05:19.498056 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489283    2582 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:05:19.498056 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489285    2582 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:05:19.498056 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489289    2582 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:05:19.498056 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489292    2582 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:05:19.498056 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489295    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:05:19.498056 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489297    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:05:19.498056 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489300    2582 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:05:19.498056 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489302    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:05:19.498056 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489305    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:05:19.498056 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489308    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:05:19.498056 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489310    2582 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:05:19.498056 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489313    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:05:19.498056 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489315    2582 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:05:19.498056 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489318    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:05:19.498056 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489321    2582 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:05:19.498056 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489323    2582 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:05:19.498056 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489326    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:05:19.498056 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489329    2582 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:05:19.498056 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489331    2582 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:05:19.498501 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489334    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:05:19.498501 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489336    2582 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:05:19.498501 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489339    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:05:19.498501 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489342    2582 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:05:19.498501 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489344    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:05:19.498501 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489347    2582 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:05:19.498501 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489349    2582 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:05:19.498501 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489352    2582 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:05:19.498501 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489354    2582 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:05:19.498501 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489357    2582 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:05:19.498501 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489360    2582 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:05:19.498501 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489363    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:05:19.498501 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489366    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:05:19.498501 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489369    2582 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:05:19.498501 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489371    2582 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:05:19.498501 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489375    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:05:19.498501 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489378    2582 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:05:19.498501 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489381    2582 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:05:19.498501 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489383    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:05:19.498501 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489386    2582 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:05:19.498998 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489389    2582 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:05:19.498998 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489391    2582 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:05:19.498998 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489394    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:05:19.498998 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489396    2582 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:05:19.498998 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489399    2582 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:05:19.498998 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489402    2582 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:05:19.498998 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489404    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:05:19.498998 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489407    2582 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:05:19.498998 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489409    2582 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:05:19.498998 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489412    2582 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:05:19.498998 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489415    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:05:19.498998 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489417    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:05:19.498998 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489420    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:05:19.498998 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489422    2582 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:05:19.498998 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.489448    2582 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 02:05:19.498998 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489486    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:05:19.499378 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489490    2582 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:05:19.499378 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489493    2582 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:05:19.499378 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489496    2582 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:05:19.499378 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489499    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:05:19.499378 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489502    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:05:19.499378 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489505    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:05:19.499378 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489507    2582 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:05:19.499378 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489511    2582 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:05:19.499378 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489513    2582 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:05:19.499378 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489516    2582 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:05:19.499378 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489518    2582 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:05:19.499378 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489521    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:05:19.499378 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489525    2582 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:05:19.499378 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489527    2582 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:05:19.499378 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489530    2582 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:05:19.499378 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489532    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:05:19.499378 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489535    2582 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:05:19.499378 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489538    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:05:19.499378 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489540    2582 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:05:19.499378 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489543    2582 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:05:19.499874 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489546    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:05:19.499874 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489548    2582 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:05:19.499874 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489551    2582 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:05:19.499874 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489553    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:05:19.499874 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489556    2582 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:05:19.499874 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489558    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:05:19.499874 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489562    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:05:19.499874 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489565    2582 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:05:19.499874 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489569    2582 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:05:19.499874 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489573    2582 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:05:19.499874 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489577    2582 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:05:19.499874 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489580    2582 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:05:19.499874 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489583    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:05:19.499874 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489586    2582 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:05:19.499874 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489589    2582 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:05:19.499874 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489592    2582 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:05:19.499874 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489594    2582 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:05:19.499874 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489597    2582 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:05:19.500303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489599    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:05:19.500303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489603    2582 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:05:19.500303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489606    2582 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:05:19.500303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489609    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:05:19.500303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489611    2582 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:05:19.500303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489614    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:05:19.500303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489618    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:05:19.500303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489621    2582 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:05:19.500303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489623    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:05:19.500303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489626    2582 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:05:19.500303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489628    2582 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:05:19.500303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489631    2582 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:05:19.500303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489633    2582 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:05:19.500303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489636    2582 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:05:19.500303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489639    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:05:19.500303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489641    2582 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:05:19.500303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489644    2582 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:05:19.500303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489646    2582 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:05:19.500303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489649    2582 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:05:19.500303 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489651    2582 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:05:19.500837 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489654    2582 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:05:19.500837 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489657    2582 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:05:19.500837 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489659    2582 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:05:19.500837 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489661    2582 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:05:19.500837 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489664    2582 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:05:19.500837 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489666    2582 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:05:19.500837 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489669    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:05:19.500837 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489672    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:05:19.500837 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489675    2582 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:05:19.500837 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489677    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:05:19.500837 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489680    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:05:19.500837 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489682    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:05:19.500837 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489685    2582 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:05:19.500837 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489688    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:05:19.500837 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489691    2582 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:05:19.500837 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489693    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:05:19.500837 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489696    2582 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:05:19.500837 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489698    2582 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:05:19.500837 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489703    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:05:19.501346 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489705    2582 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:05:19.501346 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489708    2582 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:05:19.501346 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489711    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:05:19.501346 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489713    2582 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:05:19.501346 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489716    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:05:19.501346 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489718    2582 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:05:19.501346 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489721    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:05:19.501346 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489723    2582 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:05:19.501346 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489726    2582 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:05:19.501346 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489728    2582 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:05:19.501346 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489731    2582 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:05:19.501346 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489734    2582 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:05:19.501346 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489736    2582 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:05:19.501346 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489739    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:05:19.501346 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489741    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:05:19.501346 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489744    2582 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:05:19.501346 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489747    2582 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:05:19.501346 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489749    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:05:19.501346 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489752    2582 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:05:19.501346 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489755    2582 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:05:19.501815 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489757    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:05:19.501815 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489761    2582 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:05:19.501815 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489763    2582 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:05:19.501815 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489766    2582 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:05:19.501815 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489769    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:05:19.501815 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489772    2582 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:05:19.501815 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489776    2582 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:05:19.501815 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.489778    2582 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:05:19.501815 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.495583    2582 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 04 02:05:19.501815 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.495599    2582 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 04 02:05:19.501815 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.495614    2582 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 04 02:05:19.501815 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.495619    2582 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 04 02:05:19.501815 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495694    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:05:19.501815 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495699    2582 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:05:19.501815 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495702    2582 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:05:19.501815 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495705    2582 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:05:19.501815 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495708    2582 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:05:19.501815 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495711    2582 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:05:19.501815 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495714    2582 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:05:19.502629 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495716    2582 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:05:19.502629 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495719    2582 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:05:19.502629 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495721    2582 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:05:19.502629 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495724    2582 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:05:19.502629 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495727    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:05:19.502629 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495729    2582 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:05:19.502629 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495731    2582 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:05:19.502629 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495734    2582 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:05:19.502629 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495737    2582 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:05:19.502629 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495739    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:05:19.502629 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495742    2582 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:05:19.502629 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495744    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:05:19.502629 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495747    2582 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:05:19.502629 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495749    2582 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:05:19.502629 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495752    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:05:19.502629 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495754    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:05:19.502629 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495757    2582 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:05:19.502629 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495759    2582 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:05:19.502629 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495762    2582 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:05:19.503348 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495765    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:05:19.503348 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495767    2582 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:05:19.503348 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495769    2582 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:05:19.503348 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495773    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:05:19.503348 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495776    2582 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:05:19.503348 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495779    2582 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:05:19.503348 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495781    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:05:19.503348 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495784    2582 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:05:19.503348 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495787    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:05:19.503348 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495789    2582 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:05:19.503348 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495792    2582 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:05:19.503348 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495794    2582 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:05:19.503348 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495797    2582 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:05:19.503348 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495799    2582 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:05:19.503348 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495802    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:05:19.503348 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495804    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:05:19.503348 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495807    2582 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:05:19.503348 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495809    2582 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:05:19.503348 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495812    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:05:19.503348 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495814    2582 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:05:19.503844 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495817    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:05:19.503844 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495819    2582 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:05:19.503844 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495822    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:05:19.503844 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495824    2582 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:05:19.503844 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495827    2582 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:05:19.503844 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495829    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:05:19.503844 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495832    2582 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:05:19.503844 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495834    2582 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:05:19.503844 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495837    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:05:19.503844 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495839    2582 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:05:19.503844 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495842    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:05:19.503844 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495845    2582 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:05:19.503844 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495847    2582 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:05:19.503844 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495850    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:05:19.503844 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495855    2582 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:05:19.503844 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495858    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:05:19.503844 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495862    2582 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:05:19.503844 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495865    2582 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:05:19.503844 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495867    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:05:19.504386 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495869    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:05:19.504386 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495872    2582 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:05:19.504386 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495875    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:05:19.504386 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495877    2582 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:05:19.504386 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495880    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:05:19.504386 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495882    2582 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:05:19.504386 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495899    2582 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:05:19.504386 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495902    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:05:19.504386 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495907    2582 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:05:19.504386 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495911    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:05:19.504386 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495914    2582 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:05:19.504386 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495916    2582 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:05:19.504386 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495919    2582 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:05:19.504386 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495922    2582 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:05:19.504386 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495925    2582 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:05:19.504386 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495927    2582 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:05:19.504386 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495930    2582 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:05:19.504386 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495933    2582 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:05:19.504386 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495936    2582 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:05:19.504826 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495938    2582 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:05:19.504826 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495941    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:05:19.504826 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495944    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:05:19.504826 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495946    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:05:19.504826 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495948    2582 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:05:19.504826 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495951    2582 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:05:19.504826 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495954    2582 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:05:19.504826 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495957    2582 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:05:19.504826 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495960    2582 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:05:19.504826 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495963    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:05:19.504826 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495966    2582 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:05:19.504826 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495969    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:05:19.504826 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495972    2582 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:05:19.504826 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495974    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:05:19.504826 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495977    2582 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:05:19.504826 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495979    2582 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:05:19.504826 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495982    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:05:19.504826 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495984    2582 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:05:19.504826 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495987    2582 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:05:19.504826 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495990    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:05:19.505349 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495992    2582 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:05:19.505349 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.495995    2582 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:05:19.505349 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.496022    2582 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 02:05:19.505349 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496151    2582 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:05:19.505349 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496155    2582 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:05:19.505349 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496158    2582 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:05:19.505349 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496161    2582 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:05:19.505349 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496163    2582 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:05:19.505349 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496166    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:05:19.505349 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496169    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:05:19.505349 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496171    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:05:19.505349 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496174    2582 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:05:19.505349 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496176    2582 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:05:19.505349 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496179    2582 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:05:19.505349 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496181    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:05:19.505349 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496184    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:05:19.505749 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496186    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:05:19.505749 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496189    2582 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:05:19.505749 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496192    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:05:19.505749 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496195    2582 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:05:19.505749 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496197    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:05:19.505749 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496200    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:05:19.505749 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496202    2582 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:05:19.505749 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496205    2582 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:05:19.505749 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496208    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:05:19.505749 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496210    2582 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:05:19.505749 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496213    2582 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:05:19.505749 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496216    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:05:19.505749 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496218    2582 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:05:19.505749 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496221    2582 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:05:19.505749 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496223    2582 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:05:19.505749 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496226    2582 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:05:19.505749 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496228    2582 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:05:19.505749 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496231    2582 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:05:19.505749 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496233    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:05:19.505749 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496235    2582 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:05:19.506274 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496238    2582 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:05:19.506274 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496240    2582 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:05:19.506274 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496243    2582 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:05:19.506274 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496245    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:05:19.506274 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496248    2582 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:05:19.506274 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496251    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:05:19.506274 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496254    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:05:19.506274 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496256    2582 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:05:19.506274 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496258    2582 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:05:19.506274 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496261    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:05:19.506274 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496263    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:05:19.506274 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496266    2582 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:05:19.506274 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496268    2582 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:05:19.506274 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496271    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:05:19.506274 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496273    2582 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:05:19.506274 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496276    2582 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:05:19.506274 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496279    2582 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:05:19.506274 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496281    2582 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:05:19.506274 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496284    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:05:19.506274 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496286    2582 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:05:19.506870 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496289    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:05:19.506870 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496292    2582 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:05:19.506870 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496294    2582 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:05:19.506870 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496297    2582 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:05:19.506870 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496299    2582 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:05:19.506870 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496302    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:05:19.506870 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496304    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:05:19.506870 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496307    2582 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:05:19.506870 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496309    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:05:19.506870 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496312    2582 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:05:19.506870 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496316    2582 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:05:19.506870 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496319    2582 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:05:19.506870 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496322    2582 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:05:19.506870 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496325    2582 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:05:19.506870 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496328    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:05:19.506870 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496331    2582 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:05:19.506870 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496333    2582 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:05:19.506870 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496336    2582 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:05:19.506870 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496338    2582 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:05:19.507465 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496341    2582 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:05:19.507465 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496343    2582 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:05:19.507465 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496346    2582 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:05:19.507465 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496349    2582 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:05:19.507465 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496352    2582 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:05:19.507465 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496354    2582 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:05:19.507465 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496357    2582 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:05:19.507465 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496360    2582 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:05:19.507465 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496364    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:05:19.507465 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496367    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:05:19.507465 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496370    2582 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:05:19.507465 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496374    2582 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:05:19.507465 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496376    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:05:19.507465 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496379    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:05:19.507465 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496382    2582 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:05:19.507465 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496385    2582 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:05:19.507465 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496387    2582 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:05:19.507465 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496390    2582 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:05:19.507465 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496392    2582 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:05:19.507941 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496394    2582 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:05:19.507941 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496397    2582 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:05:19.507941 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496399    2582 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:05:19.507941 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496402    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:05:19.507941 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496404    2582 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:05:19.507941 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496407    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:05:19.507941 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496409    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:05:19.507941 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496411    2582 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:05:19.507941 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496414    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:05:19.507941 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496416    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:05:19.507941 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496419    2582 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:05:19.507941 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496421    2582 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:05:19.507941 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496424    2582 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:05:19.507941 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496426    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:05:19.507941 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496429    2582 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:05:19.507941 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.496453    2582 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 02:05:19.508340 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496482    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:05:19.508340 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496486    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:05:19.508340 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496489    2582 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:05:19.508340 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496492    2582 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:05:19.508340 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496495    2582 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:05:19.508340 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496497    2582 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:05:19.508340 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496500    2582 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:05:19.508340 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496502    2582 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:05:19.508340 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496505    2582 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:05:19.508340 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496508    2582 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:05:19.508340 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496510    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:05:19.508340 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496513    2582 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:05:19.508340 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496516    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:05:19.508340 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496519    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:05:19.508340 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496521    2582 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:05:19.508340 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496524    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:05:19.508340 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496526    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:05:19.508340 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496529    2582 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:05:19.508340 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496532    2582 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:05:19.508873 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496534    2582 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:05:19.508873 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496537    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:05:19.508873 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496540    2582 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:05:19.508873 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496543    2582 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:05:19.508873 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496546    2582 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:05:19.508873 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496549    2582 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:05:19.508873 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496551    2582 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:05:19.508873 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496554    2582 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:05:19.508873 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496556    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:05:19.508873 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496559    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:05:19.508873 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496564    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:05:19.508873 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496567    2582 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:05:19.508873 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496569    2582 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:05:19.508873 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496571    2582 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:05:19.508873 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496574    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:05:19.508873 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496577    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:05:19.508873 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496579    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:05:19.508873 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496581    2582 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:05:19.508873 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496584    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:05:19.508873 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496587    2582 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:05:19.509438 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496590    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:05:19.509438 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496592    2582 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:05:19.509438 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496595    2582 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:05:19.509438 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496597    2582 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:05:19.509438 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496600    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:05:19.509438 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496602    2582 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:05:19.509438 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496605    2582 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:05:19.509438 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496607    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:05:19.509438 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496610    2582 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:05:19.509438 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496612    2582 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:05:19.509438 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496615    2582 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:05:19.509438 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496617    2582 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:05:19.509438 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496620    2582 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:05:19.509438 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496622    2582 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:05:19.509438 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496625    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:05:19.509438 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496627    2582 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:05:19.509438 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496629    2582 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:05:19.509438 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496632    2582 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:05:19.509438 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496634    2582 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:05:19.509438 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496637    2582 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:05:19.509922 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496639    2582 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:05:19.509922 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496642    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:05:19.509922 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496647    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:05:19.509922 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496649    2582 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:05:19.509922 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496652    2582 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:05:19.509922 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496654    2582 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:05:19.509922 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496657    2582 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:05:19.509922 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496659    2582 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:05:19.509922 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496662    2582 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:05:19.509922 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496665    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:05:19.509922 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496667    2582 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:05:19.509922 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496670    2582 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:05:19.509922 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496673    2582 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:05:19.509922 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496675    2582 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:05:19.509922 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496678    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:05:19.509922 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496681    2582 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:05:19.509922 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496683    2582 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:05:19.509922 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496686    2582 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:05:19.509922 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496688    2582 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:05:19.510360 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496691    2582 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:05:19.510360 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496693    2582 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:05:19.510360 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496696    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:05:19.510360 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496699    2582 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:05:19.510360 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496701    2582 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:05:19.510360 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496704    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:05:19.510360 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496706    2582 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:05:19.510360 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496710    2582 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:05:19.510360 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496714    2582 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:05:19.510360 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496716    2582 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:05:19.510360 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496719    2582 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:05:19.510360 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496722    2582 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:05:19.510360 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496724    2582 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:05:19.510360 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496727    2582 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:05:19.510360 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496729    2582 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:05:19.510360 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496733    2582 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:05:19.510360 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496736    2582 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:05:19.510360 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496738    2582 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:05:19.510360 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496741    2582 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:05:19.510791 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496743    2582 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:05:19.510791 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496746    2582 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:05:19.510791 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496748    2582 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:05:19.510791 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496751    2582 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:05:19.510791 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496755    2582 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:05:19.510791 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496758    2582 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:05:19.510791 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496761    2582 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:05:19.510791 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496763    2582 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:05:19.510791 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:19.496766    2582 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:05:19.510791 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.496819    2582 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 04 02:05:19.510791 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.498745    2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 04 02:05:19.510791 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.499574    2582 server.go:1014] "Starting client certificate rotation"
Nov 04 02:05:19.510791 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.499670    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 02:05:19.510791 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.499706    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 02:05:19.510791 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.503990    2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 02:05:19.510791 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.505788    2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 02:05:19.514546 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.514529    2582 log.go:25] "Validated CRI v1 runtime API"
Nov 04 02:05:19.517316 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.517302    2582 log.go:25] "Validated CRI v1 image API"
Nov 04 02:05:19.518088 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.518061    2582 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 04 02:05:19.522684 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.522667    2582 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 bdbc5f21-69be-4d60-877d-26481302a4ff:/dev/nvme0n1p3 ffcd1bb5-c53e-44fe-a557-229452a7ad15:/dev/nvme0n1p4]
Nov 04 02:05:19.522732 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.522684    2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 04 02:05:19.527595 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.527495    2582 manager.go:217] Machine: {Timestamp:2026-11-04 02:05:19.526280814 +0000 UTC m=+0.296574507 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3090363 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22b7e5da8137204d2fc9ce09009a9f SystemUUID:ec22b7e5-da81-3720-4d2f-c9ce09009a9f BootID:93c774cc-f5a6-4440-aa40-e5480a1c729b Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:46:67:df:df:cf Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:46:67:df:df:cf Speed:0 Mtu:9001} {Name:ovs-system MacAddress:aa:86:bb:8b:a1:11 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 04 02:05:19.528126 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.528116    2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 04 02:05:19.528223 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.528211    2582 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 04 02:05:19.530941 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.530920    2582 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 04 02:05:19.531063 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.530942    2582 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-63.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 04 02:05:19.531657 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.531647    2582 topology_manager.go:143] "Creating topology manager with none policy"
Nov 04 02:05:19.531690 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.531659    2582 container_manager_linux.go:311] "Creating device plugin manager"
Nov 04 02:05:19.531720 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.531696    2582 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 02:05:19.531720 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.531711    2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 02:05:19.532421 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.532411    2582 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 04 02:05:19.535951 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.535939    2582 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 04 02:05:19.536063 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.536054    2582 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 04 02:05:19.536662 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.536654    2582 kubelet.go:493] "Attempting to sync node with API server"
Nov 04 02:05:19.536698 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.536667    2582 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 04 02:05:19.536698 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.536678    2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 04 02:05:19.536698 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.536686    2582 kubelet.go:405] "Adding apiserver pod source"
Nov 04 02:05:19.536698 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.536694    2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 04 02:05:19.536976 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.536965    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 02:05:19.538874 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.538850    2582 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 04 02:05:19.544995 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.544975    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 02:05:19.545701 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.545689    2582 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 04 02:05:19.545742 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.545718    2582 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 04 02:05:19.545862 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.545851    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 04 02:05:19.545920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.545865    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 04 02:05:19.545920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.545871    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 04 02:05:19.545920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.545876    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 04 02:05:19.545920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.545882    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 04 02:05:19.545920 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.545915    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 04 02:05:19.546357 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.546348    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 04 02:05:19.546385 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.546357    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 04 02:05:19.546385 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.546365    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 04 02:05:19.546385 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.546371    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 04 02:05:19.546385 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.546379    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 04 02:05:19.546537 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.546387    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 04 02:05:19.547621 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.547612    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 04 02:05:19.547655 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.547622    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 04 02:05:19.551747 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.551733    2582 server.go:1263] "Started kubelet"
Nov 04 02:05:19.551838 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.551811    2582 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 04 02:05:19.551874 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.551820    2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 04 02:05:19.551921 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.551901    2582 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 04 02:05:19.552411 ip-10-0-136-63 systemd[1]: Started Kubernetes Kubelet.
Nov 04 02:05:19.553089 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.553073    2582 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 04 02:05:19.553502 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.553490    2582 server.go:317] "Adding debug handlers to kubelet server"
Nov 04 02:05:19.557699 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.557684    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 04 02:05:19.559335 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.559312    2582 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 04 02:05:19.559428 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:19.559407    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-63.us-west-1.compute.internal\" not found"
Nov 04 02:05:19.559428 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:19.559415    2582 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 04 02:05:19.560099 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.559975    2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 04 02:05:19.560099 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.559976    2582 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 04 02:05:19.560099 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.560000    2582 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 04 02:05:19.560099 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.560069    2582 reconstruct.go:97] "Volume reconstruction finished"
Nov 04 02:05:19.560099 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.560079    2582 reconciler.go:29] "Reconciler: start to sync state"
Nov 04 02:05:19.560277 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.560269    2582 factory.go:55] Registering systemd factory
Nov 04 02:05:19.560309 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.560283    2582 factory.go:223] Registration of the systemd container factory successfully
Nov 04 02:05:19.560478 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.560466    2582 factory.go:153] Registering CRI-O factory
Nov 04 02:05:19.560538 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.560482    2582 factory.go:223] Registration of the crio container factory successfully
Nov 04 02:05:19.560538 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.560524    2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 04 02:05:19.560538 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.560538    2582 factory.go:103] Registering Raw factory
Nov 04 02:05:19.560654 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.560548    2582 manager.go:1199] Started watching for new ooms in manager
Nov 04 02:05:19.561444 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.561427    2582 manager.go:319] Starting recovery of all containers
Nov 04 02:05:19.570975 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.570879    2582 manager.go:324] Recovery completed
Nov 04 02:05:19.577173 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.577154    2582 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 02:05:19.578737 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.578709    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 02:05:19.578804 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.578753    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 02:05:19.578804 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.578770    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 02:05:19.579274 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.579261    2582 cpu_manager.go:226] "Starting" policy="none"
Nov 04 02:05:19.579274 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.579272    2582 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 04 02:05:19.579348 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.579288    2582 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 04 02:05:19.579817 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.579799    2582 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 02:05:19.580546 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.580530    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 02:05:19.580605 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.580556    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 02:05:19.580605 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.580567    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 02:05:19.581376 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.581363    2582 policy_none.go:50] "Start"
Nov 04 02:05:19.581376 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.581379    2582 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 04 02:05:19.581461 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.581388    2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 04 02:05:19.583227 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.583217    2582 policy_none.go:44] "Start"
Nov 04 02:05:19.612154 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.612118    2582 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 04 02:05:19.614421 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.614408    2582 manager.go:341] "Starting Device Plugin manager"
Nov 04 02:05:19.614491 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:19.614444    2582 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 04 02:05:19.614491 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.614456    2582 server.go:87] "Starting device plugin registration server"
Nov 04 02:05:19.614671 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.614659    2582 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 04 02:05:19.614725 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.614673    2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 04 02:05:19.614781 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.614766    2582 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 04 02:05:19.615038 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.614900    2582 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 04 02:05:19.615038 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.614919    2582 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 04 02:05:19.615222 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:19.615204    2582 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 04 02:05:19.615302 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:19.615249    2582 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-63.us-west-1.compute.internal\" not found"
Nov 04 02:05:19.687786 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.687753    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 04 02:05:19.688844 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.688823    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 04 02:05:19.688928 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.688853    2582 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 04 02:05:19.688928 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.688873    2582 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 04 02:05:19.689015 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:19.688926    2582 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Nov 04 02:05:19.697168 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.697148    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-63.us-west-1.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 04 02:05:19.697252 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:19.697169    2582 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-63.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Nov 04 02:05:19.698019 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:19.697151    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-63.us-west-1.compute.internal.18e4b552f7ec4d3a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-63.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-63.us-west-1.compute.internal,},FirstTimestamp:2026-11-04 02:05:19.551704378 +0000 UTC m=+0.321998071,LastTimestamp:2026-11-04 02:05:19.551704378 +0000 UTC m=+0.321998071,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-63.us-west-1.compute.internal,}"
Nov 04 02:05:19.715527 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.715513    2582 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 02:05:19.716294 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.716279    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 02:05:19.716350 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.716310    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 02:05:19.716350 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.716324    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 02:05:19.716350 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.716348    2582 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:05:19.784873 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.784737    2582 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 04 02:05:19.789855 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.789839    2582 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal"]
Nov 04 02:05:19.789916 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.789896    2582 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 02:05:19.790546 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.790532    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 02:05:19.790604 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.790556    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 02:05:19.790604 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.790572    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 02:05:19.792713 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.792702    2582 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 02:05:19.792877 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.792864    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:05:19.792924 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.792909    2582 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 02:05:19.793867 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.793854    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 02:05:19.793926 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.793877    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 02:05:19.793926 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.793905    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 02:05:19.794542 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.794527    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 02:05:19.794608 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.794583    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 02:05:19.794608 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.794597    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 02:05:19.796650 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.796635    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:05:19.796711 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.796665    2582 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 02:05:19.797339 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.797324    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 02:05:19.797416 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.797348    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 02:05:19.797416 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.797358    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 02:05:19.820077 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:19.820061    2582 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-63.us-west-1.compute.internal\" not found" node="ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:05:19.822727 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:19.822711    2582 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-63.us-west-1.compute.internal\" not found" node="ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:05:19.854986 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:19.854863    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-63.us-west-1.compute.internal.18e4b552f988d42e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-63.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-63.us-west-1.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-63.us-west-1.compute.internal,},FirstTimestamp:2026-11-04 02:05:19.578739758 +0000 UTC m=+0.349033457,LastTimestamp:2026-11-04 02:05:19.578739758 +0000 UTC m=+0.349033457,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-63.us-west-1.compute.internal,}"
Nov 04 02:05:19.856287 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:19.856258    2582 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Nov 04 02:05:19.856287 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:19.856259    2582 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-63.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Nov 04 02:05:19.856401 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:19.856260    2582 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Nov 04 02:05:19.861684 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.861667    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5dd9721b3a00168bde79d6cad3060599-config\") pod \"kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal\" (UID: \"5dd9721b3a00168bde79d6cad3060599\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:05:19.861747 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.861691    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1ec35563e4041daa397c9c3ffc626b1c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal\" (UID: \"1ec35563e4041daa397c9c3ffc626b1c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:05:19.861747 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.861707    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ec35563e4041daa397c9c3ffc626b1c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal\" (UID: \"1ec35563e4041daa397c9c3ffc626b1c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:05:19.962167 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.962144    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ec35563e4041daa397c9c3ffc626b1c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal\" (UID: \"1ec35563e4041daa397c9c3ffc626b1c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:05:19.962244 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.962174    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5dd9721b3a00168bde79d6cad3060599-config\") pod \"kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal\" (UID: \"5dd9721b3a00168bde79d6cad3060599\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:05:19.962244 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.962189    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1ec35563e4041daa397c9c3ffc626b1c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal\" (UID: \"1ec35563e4041daa397c9c3ffc626b1c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:05:19.962244 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.962238    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1ec35563e4041daa397c9c3ffc626b1c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal\" (UID: \"1ec35563e4041daa397c9c3ffc626b1c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:05:19.962336 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.962245    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ec35563e4041daa397c9c3ffc626b1c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal\" (UID: \"1ec35563e4041daa397c9c3ffc626b1c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:05:19.962336 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:19.962257    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5dd9721b3a00168bde79d6cad3060599-config\") pod \"kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal\" (UID: \"5dd9721b3a00168bde79d6cad3060599\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:05:20.019008 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:20.018988    2582 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-63.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="400ms"
Nov 04 02:05:20.019080 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:20.019025    2582 kubelet_node_status.go:115] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-136-63.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:05:20.019119 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:20.019026    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-63.us-west-1.compute.internal.18e4b552f9892506  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-63.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-63.us-west-1.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-63.us-west-1.compute.internal,},FirstTimestamp:2026-11-04 02:05:19.578760454 +0000 UTC m=+0.349054149,LastTimestamp:2026-11-04 02:05:19.578760454 +0000 UTC m=+0.349054149,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-63.us-west-1.compute.internal,}"
Nov 04 02:05:20.019289 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:20.019273    2582 reflector.go:204] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass"
Nov 04 02:05:20.123160 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:20.123119    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:05:20.126197 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:20.126182    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:05:20.183831 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:20.183759    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-63.us-west-1.compute.internal.18e4b552f9896597  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-63.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-136-63.us-west-1.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-136-63.us-west-1.compute.internal,},FirstTimestamp:2026-11-04 02:05:19.578776983 +0000 UTC m=+0.349070680,LastTimestamp:2026-11-04 02:05:19.578776983 +0000 UTC m=+0.349070680,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-63.us-west-1.compute.internal,}"
Nov 04 02:05:20.219573 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:20.219557    2582 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 02:05:20.220340 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:20.220325    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 02:05:20.220399 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:20.220352    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 02:05:20.220399 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:20.220361    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 02:05:20.220399 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:20.220386    2582 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:05:20.346171 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:20.346103    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-136-63.us-west-1.compute.internal.18e4b552f988d42e\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-63.us-west-1.compute.internal.18e4b552f988d42e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-63.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-63.us-west-1.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-63.us-west-1.compute.internal,},FirstTimestamp:2026-11-04 02:05:19.578739758 +0000 UTC m=+0.349033457,LastTimestamp:2026-11-04 02:05:19.580543968 +0000 UTC m=+0.350837660,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-63.us-west-1.compute.internal,}"
Nov 04 02:05:21.800881 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:21.800853    2582 kubelet_node_status.go:115] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-136-63.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:05:21.800881 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:21.800865    2582 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-63.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="800ms"
Nov 04 02:05:21.801295 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:21.800888    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-136-63.us-west-1.compute.internal.18e4b552f9892506\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-63.us-west-1.compute.internal.18e4b552f9892506  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-63.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-63.us-west-1.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-63.us-west-1.compute.internal,},FirstTimestamp:2026-11-04 02:05:19.578760454 +0000 UTC m=+0.349054149,LastTimestamp:2026-11-04 02:05:19.580560357 +0000 UTC m=+0.350854049,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-63.us-west-1.compute.internal,}"
Nov 04 02:05:21.869258 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:21.869194    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-136-63.us-west-1.compute.internal.18e4b552f9896597\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-63.us-west-1.compute.internal.18e4b552f9896597  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-63.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-136-63.us-west-1.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-136-63.us-west-1.compute.internal,},FirstTimestamp:2026-11-04 02:05:19.578776983 +0000 UTC m=+0.349070680,LastTimestamp:2026-11-04 02:05:19.580572615 +0000 UTC m=+0.350866307,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-63.us-west-1.compute.internal,}"
Nov 04 02:05:21.958751 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:21.958676    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-63.us-west-1.compute.internal.18e4b552fba8c67e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-63.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated limits on system reserved cgroup /system.slice,Source:EventSource{Component:kubelet,Host:ip-10-0-136-63.us-west-1.compute.internal,},FirstTimestamp:2026-11-04 02:05:19.614387838 +0000 UTC m=+0.384681517,LastTimestamp:2026-11-04 02:05:19.614387838 +0000 UTC m=+0.384681517,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-63.us-west-1.compute.internal,}"
Nov 04 02:05:22.030317 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:22.030233    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-63.us-west-1.compute.internal.18e4b552fc7d745d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-63.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-136-63.us-west-1.compute.internal,},FirstTimestamp:2026-11-04 02:05:19.628325981 +0000 UTC m=+0.398619676,LastTimestamp:2026-11-04 02:05:19.628325981 +0000 UTC m=+0.398619676,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-63.us-west-1.compute.internal,}"
Nov 04 02:05:22.044726 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.044706    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gtm7k"
Nov 04 02:05:22.057997 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.057976    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gtm7k"
Nov 04 02:05:22.121373 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.121352    2582 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 02:05:22.128462 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.128445    2582 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 02:05:22.129198 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.129177    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 02:05:22.129277 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.129208    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 02:05:22.129277 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.129218    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 02:05:22.171686 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.171665    2582 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-136-63.us-west-1.compute.internal" not found
Nov 04 02:05:22.201262 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.201245    2582 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 02:05:22.201982 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.201970    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 02:05:22.202041 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.201993    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 02:05:22.202041 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.202004    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 02:05:22.202041 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.202025    2582 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:05:22.234237 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.234219    2582 nodeinfomanager.go:422] Failed to publish CSINode: nodes "ip-10-0-136-63.us-west-1.compute.internal" not found
Nov 04 02:05:22.260792 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.260777    2582 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:05:22.260874 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:22.260795    2582 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-63.us-west-1.compute.internal\": node \"ip-10-0-136-63.us-west-1.compute.internal\" not found"
Nov 04 02:05:22.370981 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:22.370929    2582 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-63.us-west-1.compute.internal\" not found"
Nov 04 02:05:22.373792 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.373778    2582 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 02:05:22.452053 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.452032    2582 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:05:22.547949 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:22.547920    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dd9721b3a00168bde79d6cad3060599.slice/crio-1364fc7f796bc4973dec36326a38998ec7c014e3020006690e2980334e6c5e57 WatchSource:0}: Error finding container 1364fc7f796bc4973dec36326a38998ec7c014e3020006690e2980334e6c5e57: Status 404 returned error can't find the container with id 1364fc7f796bc4973dec36326a38998ec7c014e3020006690e2980334e6c5e57
Nov 04 02:05:22.548521 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:22.548505    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ec35563e4041daa397c9c3ffc626b1c.slice/crio-687ec26b7041d8e6fda8f2619863e7784224c00106904fb64950ce9f99151adc WatchSource:0}: Error finding container 687ec26b7041d8e6fda8f2619863e7784224c00106904fb64950ce9f99151adc: Status 404 returned error can't find the container with id 687ec26b7041d8e6fda8f2619863e7784224c00106904fb64950ce9f99151adc
Nov 04 02:05:22.554956 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.554938    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 02:05:22.582127 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.582109    2582 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 02:05:22.583760 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.583747    2582 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:05:22.677836 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.677792    2582 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 02:05:22.758848 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.758828    2582 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 02:05:22.791766 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.791749    2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 04 02:05:22.791917 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.791898    2582 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 04 02:05:22.791968 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.791898    2582 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 04 02:05:22.791968 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:22.791895    2582 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://ac66bc7e1762247a7b68d1e1f0c05235-88944398bf9cb6fa.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-136-63.us-west-1.compute.internal.18e4b552f988d42e\": read tcp 10.0.136.63:50690->18.119.206.145:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-136-63.us-west-1.compute.internal.18e4b552f988d42e  default   4805 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-63.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-63.us-west-1.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-63.us-west-1.compute.internal,},FirstTimestamp:2026-11-04 02:05:19 +0000 UTC,LastTimestamp:2026-11-04 02:05:19.79454171 +0000 UTC m=+0.564835412,Count:6,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-63.us-west-1.compute.internal,}"
Nov 04 02:05:22.830729 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.830710    2582 apiserver.go:52] "Watching apiserver"
Nov 04 02:05:22.850073 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.850054    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 04 02:05:22.984785 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.984722    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal" event={"ID":"1ec35563e4041daa397c9c3ffc626b1c","Type":"ContainerStarted","Data":"687ec26b7041d8e6fda8f2619863e7784224c00106904fb64950ce9f99151adc"}
Nov 04 02:05:22.985803 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:22.985785    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal" event={"ID":"5dd9721b3a00168bde79d6cad3060599","Type":"ContainerStarted","Data":"1364fc7f796bc4973dec36326a38998ec7c014e3020006690e2980334e6c5e57"}
Nov 04 02:05:23.061408 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.061368    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 02:00:22 +0000 UTC" deadline="2028-08-13 03:09:21.997422378 +0000 UTC"
Nov 04 02:05:23.061408 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.061401    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15553h3m58.936025228s"
Nov 04 02:05:23.111883 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.111864    2582 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 04 02:05:23.112182 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.112164    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-4v7xm","openshift-network-diagnostics/network-check-target-98h6z","openshift-dns/node-resolver-vjp7p","openshift-image-registry/node-ca-gzg65","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal","openshift-network-operator/iptables-alerter-qvk5m","kube-system/kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb","openshift-cluster-node-tuning-operator/tuned-98zkj","openshift-ovn-kubernetes/ovnkube-node-2l29t","kube-system/global-pull-secret-syncer-b4wjr","kube-system/konnectivity-agent-928nm","openshift-multus/multus-additional-cni-plugins-jsh5m","openshift-multus/multus-hclbr"]
Nov 04 02:05:23.115744 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.115727    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:05:23.115829 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:23.115809    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v7xm" podUID="afe4b3d8-a18c-4587-9057-83123ffe3347"
Nov 04 02:05:23.117811 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.117794    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:05:23.117864 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:23.117849    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98h6z" podUID="b0a72217-fe1d-4589-9201-651503694cb4"
Nov 04 02:05:23.119874 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.119857    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vjp7p"
Nov 04 02:05:23.121874 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.121858    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gzg65"
Nov 04 02:05:23.125063 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.125043    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qvk5m"
Nov 04 02:05:23.127233 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.127214    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:05:23.129364 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.129350    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.131670 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.131657    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.133903 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.133888    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:23.133984 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:23.133963    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4wjr" podUID="e164905d-5505-4d20-af2e-b8acd370bf18"
Nov 04 02:05:23.136025 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.136010    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-928nm"
Nov 04 02:05:23.138434 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.138418    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:05:23.141411 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.141394    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.152897 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.152880    2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 04 02:05:23.166282 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.166263    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-slash\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.166360 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.166286    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-run\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.166360 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.166301    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a395e26-3eac-4728-bbad-c42eaf6da56d-cni-binary-copy\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:05:23.166360 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.166316    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-cnibin\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.166360 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.166332    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-multus-daemon-config\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.166530 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.166371    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-482t2\" (UniqueName: \"kubernetes.io/projected/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-kube-api-access-482t2\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:05:23.166530 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.166422    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/39235bb9-36b4-43e4-9161-043a3fb6a261-agent-certs\") pod \"konnectivity-agent-928nm\" (UID: \"39235bb9-36b4-43e4-9161-043a3fb6a261\") " pod="kube-system/konnectivity-agent-928nm"
Nov 04 02:05:23.166530 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.166460    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-kubernetes\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.166530 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.166498    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-run-systemd\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.166530 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.166524    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-etc-openvswitch\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.166750 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.166549    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-log-socket\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.166750 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.166598    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcxzt\" (UniqueName: \"kubernetes.io/projected/8ddd2cf3-5cce-41a8-ad26-6946d3cb5910-kube-api-access-lcxzt\") pod \"node-ca-gzg65\" (UID: \"8ddd2cf3-5cce-41a8-ad26-6946d3cb5910\") " pod="openshift-image-registry/node-ca-gzg65"
Nov 04 02:05:23.166750 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.166626    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-socket-dir\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:05:23.166750 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.166653    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcv2t\" (UniqueName: \"kubernetes.io/projected/70aa0113-0423-4bdf-8335-d6c5bd707757-kube-api-access-fcv2t\") pod \"node-resolver-vjp7p\" (UID: \"70aa0113-0423-4bdf-8335-d6c5bd707757\") " pod="openshift-dns/node-resolver-vjp7p"
Nov 04 02:05:23.166750 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.166678    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-system-cni-dir\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.166750 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.166701    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-multus-cni-dir\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.166750 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.166723    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-run-k8s-cni-cncf-io\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.166750 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.166747    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-var-lib-kubelet\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.167084 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.166769    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-multus-conf-dir\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.167084 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.166795    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e164905d-5505-4d20-af2e-b8acd370bf18-original-pull-secret\") pod \"global-pull-secret-syncer-b4wjr\" (UID: \"e164905d-5505-4d20-af2e-b8acd370bf18\") " pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:23.167084 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.166818    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-cni-netd\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.167084 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.166842    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a395e26-3eac-4728-bbad-c42eaf6da56d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:05:23.167084 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.166866    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84hdg\" (UniqueName: \"kubernetes.io/projected/9a395e26-3eac-4728-bbad-c42eaf6da56d-kube-api-access-84hdg\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:05:23.167084 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.166898    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/16c89b3c-f63f-4489-8eae-5918677e1e61-host-slash\") pod \"iptables-alerter-qvk5m\" (UID: \"16c89b3c-f63f-4489-8eae-5918677e1e61\") " pod="openshift-network-operator/iptables-alerter-qvk5m"
Nov 04 02:05:23.167084 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.166935    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85d9k\" (UniqueName: \"kubernetes.io/projected/16c89b3c-f63f-4489-8eae-5918677e1e61-kube-api-access-85d9k\") pod \"iptables-alerter-qvk5m\" (UID: \"16c89b3c-f63f-4489-8eae-5918677e1e61\") " pod="openshift-network-operator/iptables-alerter-qvk5m"
Nov 04 02:05:23.167084 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.166979    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:05:23.167084 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167011    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/70aa0113-0423-4bdf-8335-d6c5bd707757-hosts-file\") pod \"node-resolver-vjp7p\" (UID: \"70aa0113-0423-4bdf-8335-d6c5bd707757\") " pod="openshift-dns/node-resolver-vjp7p"
Nov 04 02:05:23.167084 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167038    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-sysconfig\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.167084 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167063    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-systemd\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.167682 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167086    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-lib-modules\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.167682 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167127    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-run-multus-certs\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.167682 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167150    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-cni-binary-copy\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.167682 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167170    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-host\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.167682 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167204    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-ovn-node-metrics-cert\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.167682 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167233    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a395e26-3eac-4728-bbad-c42eaf6da56d-cnibin\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:05:23.167682 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167259    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-etc-selinux\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:05:23.167682 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167290    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e164905d-5505-4d20-af2e-b8acd370bf18-kubelet-config\") pod \"global-pull-secret-syncer-b4wjr\" (UID: \"e164905d-5505-4d20-af2e-b8acd370bf18\") " pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:23.167682 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167318    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrkqj\" (UniqueName: \"kubernetes.io/projected/b0a72217-fe1d-4589-9201-651503694cb4-kube-api-access-nrkqj\") pod \"network-check-target-98h6z\" (UID: \"b0a72217-fe1d-4589-9201-651503694cb4\") " pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:05:23.167682 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167351    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-var-lib-kubelet\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.167682 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167388    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-ovnkube-script-lib\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.167682 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167412    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-run-netns\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.167682 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167435    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-var-lib-cni-bin\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.167682 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167471    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e164905d-5505-4d20-af2e-b8acd370bf18-dbus\") pod \"global-pull-secret-syncer-b4wjr\" (UID: \"e164905d-5505-4d20-af2e-b8acd370bf18\") " pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:23.167682 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167511    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-var-lib-cni-multus\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.167682 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167532    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-modprobe-d\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.168222 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167556    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-sysctl-conf\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.168222 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167595    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-etc-kubernetes\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.168222 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167627    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/16c89b3c-f63f-4489-8eae-5918677e1e61-iptables-alerter-script\") pod \"iptables-alerter-qvk5m\" (UID: \"16c89b3c-f63f-4489-8eae-5918677e1e61\") " pod="openshift-network-operator/iptables-alerter-qvk5m"
Nov 04 02:05:23.168222 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167651    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l89wj\" (UniqueName: \"kubernetes.io/projected/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-kube-api-access-l89wj\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.168222 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167673    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/39235bb9-36b4-43e4-9161-043a3fb6a261-konnectivity-ca\") pod \"konnectivity-agent-928nm\" (UID: \"39235bb9-36b4-43e4-9161-043a3fb6a261\") " pod="kube-system/konnectivity-agent-928nm"
Nov 04 02:05:23.168222 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167693    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70aa0113-0423-4bdf-8335-d6c5bd707757-tmp-dir\") pod \"node-resolver-vjp7p\" (UID: \"70aa0113-0423-4bdf-8335-d6c5bd707757\") " pod="openshift-dns/node-resolver-vjp7p"
Nov 04 02:05:23.168222 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167739    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.168222 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167769    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-multus-socket-dir-parent\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.168222 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167803    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs\") pod \"network-metrics-daemon-4v7xm\" (UID: \"afe4b3d8-a18c-4587-9057-83123ffe3347\") " pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:05:23.168222 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167830    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-device-dir\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:05:23.168222 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167862    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-tmp\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.168222 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167899    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-systemd-units\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.168222 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167922    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8ddd2cf3-5cce-41a8-ad26-6946d3cb5910-serviceca\") pod \"node-ca-gzg65\" (UID: \"8ddd2cf3-5cce-41a8-ad26-6946d3cb5910\") " pod="openshift-image-registry/node-ca-gzg65"
Nov 04 02:05:23.168222 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167937    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-registration-dir\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:05:23.168222 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.167978    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-var-lib-openvswitch\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.168222 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.168001    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-sysctl-d\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.168852 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.168023    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-env-overrides\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.168852 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.168046    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a395e26-3eac-4728-bbad-c42eaf6da56d-system-cni-dir\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:05:23.168852 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.168072    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-sys\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.168852 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.168109    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-tuned\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.168852 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.168131    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pl9r\" (UniqueName: \"kubernetes.io/projected/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-kube-api-access-9pl9r\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.168852 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.168152    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-kubelet\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.168852 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.168190    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-run-ovn-kubernetes\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.168852 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.168223    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9a395e26-3eac-4728-bbad-c42eaf6da56d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:05:23.168852 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.168246    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ddd2cf3-5cce-41a8-ad26-6946d3cb5910-host\") pod \"node-ca-gzg65\" (UID: \"8ddd2cf3-5cce-41a8-ad26-6946d3cb5910\") " pod="openshift-image-registry/node-ca-gzg65"
Nov 04 02:05:23.168852 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.168260    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-run-netns\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.168852 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.168294    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-run-openvswitch\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.168852 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.168329    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-run-ovn\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.168852 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.168363    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-node-log\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.168852 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.168392    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-ovnkube-config\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.168852 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.168415    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkch2\" (UniqueName: \"kubernetes.io/projected/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-kube-api-access-hkch2\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.168852 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.168444    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9a395e26-3eac-4728-bbad-c42eaf6da56d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:05:23.169446 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.168481    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-hostroot\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.169446 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.168501    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-cni-bin\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.169446 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.168532    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr84q\" (UniqueName: \"kubernetes.io/projected/afe4b3d8-a18c-4587-9057-83123ffe3347-kube-api-access-fr84q\") pod \"network-metrics-daemon-4v7xm\" (UID: \"afe4b3d8-a18c-4587-9057-83123ffe3347\") " pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:05:23.169446 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.168557    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-sys-fs\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:05:23.173733 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.173718    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ckptg\""
Nov 04 02:05:23.173796 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.173739    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 04 02:05:23.173796 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.173747    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 04 02:05:23.175040 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.175016    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z899q\""
Nov 04 02:05:23.175040 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.175016    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 04 02:05:23.175222 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.175080    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 04 02:05:23.175222 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.175141    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 04 02:05:23.177956 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.177941    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 04 02:05:23.178210 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.178196    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rtznt\""
Nov 04 02:05:23.178333 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.178320    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 04 02:05:23.178373 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.178320    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 04 02:05:23.180031 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.180017    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 04 02:05:23.180498 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.180484    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 04 02:05:23.180639 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.180623    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-h6pb6\""
Nov 04 02:05:23.180707 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.180695    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 04 02:05:23.182369 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.182353    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 04 02:05:23.182456 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.182415    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-db624\""
Nov 04 02:05:23.216205 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.216180    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 04 02:05:23.216205 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.216184    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 04 02:05:23.216343 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.216328    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 04 02:05:23.216343 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.216332    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 04 02:05:23.216415 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.216359    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 04 02:05:23.216462 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.216418    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 04 02:05:23.216462 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.216421    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 04 02:05:23.216462 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.216459    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 04 02:05:23.216580 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.216503    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qq5vw\""
Nov 04 02:05:23.220028 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.220016    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 04 02:05:23.220084 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.220074    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 04 02:05:23.220179 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.220166    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 04 02:05:23.222031 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.222020    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 04 02:05:23.228623 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.228608    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 04 02:05:23.229042 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.229015    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 04 02:05:23.229042 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.229017    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 04 02:05:23.229042 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.229036    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-kn9lr\""
Nov 04 02:05:23.230784 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.230769    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-szjlx\""
Nov 04 02:05:23.231036 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.231023    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kvkfc\""
Nov 04 02:05:23.231465 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.231450    2582 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 04 02:05:23.231536 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.231510    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ww6r2"
Nov 04 02:05:23.269680 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.269658    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-etc-openvswitch\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.269766 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.269684    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-log-socket\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.269766 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.269700    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcxzt\" (UniqueName: \"kubernetes.io/projected/8ddd2cf3-5cce-41a8-ad26-6946d3cb5910-kube-api-access-lcxzt\") pod \"node-ca-gzg65\" (UID: \"8ddd2cf3-5cce-41a8-ad26-6946d3cb5910\") " pod="openshift-image-registry/node-ca-gzg65"
Nov 04 02:05:23.269766 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.269714    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-socket-dir\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:05:23.269766 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.269731    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcv2t\" (UniqueName: \"kubernetes.io/projected/70aa0113-0423-4bdf-8335-d6c5bd707757-kube-api-access-fcv2t\") pod \"node-resolver-vjp7p\" (UID: \"70aa0113-0423-4bdf-8335-d6c5bd707757\") " pod="openshift-dns/node-resolver-vjp7p"
Nov 04 02:05:23.269766 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.269752    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-system-cni-dir\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.269998 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.269776    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-log-socket\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.269998 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.269803    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-etc-openvswitch\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.269998 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.269811    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-multus-cni-dir\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.269998 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.269847    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-run-k8s-cni-cncf-io\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.269998 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.269863    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-socket-dir\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:05:23.269998 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.269875    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-system-cni-dir\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.269998 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.269877    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-var-lib-kubelet\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.269998 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.269911    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-var-lib-kubelet\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.269998 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.269918    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-run-k8s-cni-cncf-io\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.269998 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.269926    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-multus-conf-dir\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.269998 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.269961    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e164905d-5505-4d20-af2e-b8acd370bf18-original-pull-secret\") pod \"global-pull-secret-syncer-b4wjr\" (UID: \"e164905d-5505-4d20-af2e-b8acd370bf18\") " pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:23.269998 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.269975    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-multus-cni-dir\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.269998 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.269982    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-cni-netd\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.269998 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.269974    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-multus-conf-dir\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.270640 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270016    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a395e26-3eac-4728-bbad-c42eaf6da56d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:05:23.270640 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270046    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84hdg\" (UniqueName: \"kubernetes.io/projected/9a395e26-3eac-4728-bbad-c42eaf6da56d-kube-api-access-84hdg\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:05:23.270640 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270020    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-cni-netd\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.270640 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270075    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/16c89b3c-f63f-4489-8eae-5918677e1e61-host-slash\") pod \"iptables-alerter-qvk5m\" (UID: \"16c89b3c-f63f-4489-8eae-5918677e1e61\") " pod="openshift-network-operator/iptables-alerter-qvk5m"
Nov 04 02:05:23.270640 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:23.270086    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 02:05:23.270640 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270117    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85d9k\" (UniqueName: \"kubernetes.io/projected/16c89b3c-f63f-4489-8eae-5918677e1e61-kube-api-access-85d9k\") pod \"iptables-alerter-qvk5m\" (UID: \"16c89b3c-f63f-4489-8eae-5918677e1e61\") " pod="openshift-network-operator/iptables-alerter-qvk5m"
Nov 04 02:05:23.270640 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270136    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/16c89b3c-f63f-4489-8eae-5918677e1e61-host-slash\") pod \"iptables-alerter-qvk5m\" (UID: \"16c89b3c-f63f-4489-8eae-5918677e1e61\") " pod="openshift-network-operator/iptables-alerter-qvk5m"
Nov 04 02:05:23.270640 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:23.270153    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e164905d-5505-4d20-af2e-b8acd370bf18-original-pull-secret podName:e164905d-5505-4d20-af2e-b8acd370bf18 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:23.77013409 +0000 UTC m=+3.248221844 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e164905d-5505-4d20-af2e-b8acd370bf18-original-pull-secret") pod "global-pull-secret-syncer-b4wjr" (UID: "e164905d-5505-4d20-af2e-b8acd370bf18") : object "kube-system"/"original-pull-secret" not registered
Nov 04 02:05:23.270640 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270184    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:05:23.270640 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270198    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a395e26-3eac-4728-bbad-c42eaf6da56d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:05:23.270640 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270215    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/70aa0113-0423-4bdf-8335-d6c5bd707757-hosts-file\") pod \"node-resolver-vjp7p\" (UID: \"70aa0113-0423-4bdf-8335-d6c5bd707757\") " pod="openshift-dns/node-resolver-vjp7p"
Nov 04 02:05:23.270640 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270236    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-sysconfig\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.270640 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270240    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:05:23.270640 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270252    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-systemd\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.270640 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270267    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-lib-modules\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.270640 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270277    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/70aa0113-0423-4bdf-8335-d6c5bd707757-hosts-file\") pod \"node-resolver-vjp7p\" (UID: \"70aa0113-0423-4bdf-8335-d6c5bd707757\") " pod="openshift-dns/node-resolver-vjp7p"
Nov 04 02:05:23.270640 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270291    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-run-multus-certs\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.271282 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270306    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-sysconfig\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.271282 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270317    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-cni-binary-copy\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.271282 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270323    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-systemd\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.271282 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270338    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-run-multus-certs\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.271282 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270346    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-host\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.271282 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270371    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-ovn-node-metrics-cert\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.271282 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270380    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-lib-modules\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.271282 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270395    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a395e26-3eac-4728-bbad-c42eaf6da56d-cnibin\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:05:23.271282 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270408    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-host\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.271282 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270419    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-etc-selinux\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:05:23.271282 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270443    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e164905d-5505-4d20-af2e-b8acd370bf18-kubelet-config\") pod \"global-pull-secret-syncer-b4wjr\" (UID: \"e164905d-5505-4d20-af2e-b8acd370bf18\") " pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:23.271282 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270447    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a395e26-3eac-4728-bbad-c42eaf6da56d-cnibin\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:05:23.271282 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270479    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrkqj\" (UniqueName: \"kubernetes.io/projected/b0a72217-fe1d-4589-9201-651503694cb4-kube-api-access-nrkqj\") pod \"network-check-target-98h6z\" (UID: \"b0a72217-fe1d-4589-9201-651503694cb4\") " pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:05:23.271282 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270501    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e164905d-5505-4d20-af2e-b8acd370bf18-kubelet-config\") pod \"global-pull-secret-syncer-b4wjr\" (UID: \"e164905d-5505-4d20-af2e-b8acd370bf18\") " pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:23.271282 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270515    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-var-lib-kubelet\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.271282 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270528    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-etc-selinux\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:05:23.271282 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270537    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-ovnkube-script-lib\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.272083 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270560    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-run-netns\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.272083 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270583    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-var-lib-cni-bin\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.272083 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270570    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-var-lib-kubelet\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.272083 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270611    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e164905d-5505-4d20-af2e-b8acd370bf18-dbus\") pod \"global-pull-secret-syncer-b4wjr\" (UID: \"e164905d-5505-4d20-af2e-b8acd370bf18\") " pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:23.272083 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270640    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-run-netns\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.272083 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270651    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-var-lib-cni-multus\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.272083 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270658    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e164905d-5505-4d20-af2e-b8acd370bf18-dbus\") pod \"global-pull-secret-syncer-b4wjr\" (UID: \"e164905d-5505-4d20-af2e-b8acd370bf18\") " pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:23.272083 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270673    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-modprobe-d\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.272083 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270687    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-var-lib-cni-multus\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.272083 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270689    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-sysctl-conf\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.272083 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270650    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-var-lib-cni-bin\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.272083 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270720    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-etc-kubernetes\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.272083 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270745    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/16c89b3c-f63f-4489-8eae-5918677e1e61-iptables-alerter-script\") pod \"iptables-alerter-qvk5m\" (UID: \"16c89b3c-f63f-4489-8eae-5918677e1e61\") " pod="openshift-network-operator/iptables-alerter-qvk5m"
Nov 04 02:05:23.272083 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270749    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-modprobe-d\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.272083 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270770    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l89wj\" (UniqueName: \"kubernetes.io/projected/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-kube-api-access-l89wj\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.272083 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270787    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-etc-kubernetes\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.272083 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270791    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-sysctl-conf\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.272083 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270821    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/39235bb9-36b4-43e4-9161-043a3fb6a261-konnectivity-ca\") pod \"konnectivity-agent-928nm\" (UID: \"39235bb9-36b4-43e4-9161-043a3fb6a261\") " pod="kube-system/konnectivity-agent-928nm"
Nov 04 02:05:23.272900 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270851    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70aa0113-0423-4bdf-8335-d6c5bd707757-tmp-dir\") pod \"node-resolver-vjp7p\" (UID: \"70aa0113-0423-4bdf-8335-d6c5bd707757\") " pod="openshift-dns/node-resolver-vjp7p"
Nov 04 02:05:23.272900 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270876    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.272900 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270901    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-multus-socket-dir-parent\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.272900 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270919    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs\") pod \"network-metrics-daemon-4v7xm\" (UID: \"afe4b3d8-a18c-4587-9057-83123ffe3347\") " pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:05:23.272900 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270939    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-device-dir\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:05:23.272900 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270963    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-tmp\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.272900 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.270983    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-multus-socket-dir-parent\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.272900 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:23.271016    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 02:05:23.272900 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271021    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.272900 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:23.271064    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs podName:afe4b3d8-a18c-4587-9057-83123ffe3347 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:23.77104926 +0000 UTC m=+3.249137025 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs") pod "network-metrics-daemon-4v7xm" (UID: "afe4b3d8-a18c-4587-9057-83123ffe3347") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 02:05:23.272900 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271065    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-device-dir\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:05:23.272900 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271107    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-systemd-units\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.272900 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271132    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-systemd-units\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.272900 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271136    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8ddd2cf3-5cce-41a8-ad26-6946d3cb5910-serviceca\") pod \"node-ca-gzg65\" (UID: \"8ddd2cf3-5cce-41a8-ad26-6946d3cb5910\") " pod="openshift-image-registry/node-ca-gzg65"
Nov 04 02:05:23.272900 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271166    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-registration-dir\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:05:23.272900 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271175    2582 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 04 02:05:23.272900 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271192    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-var-lib-openvswitch\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.273713 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271234    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-var-lib-openvswitch\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.273713 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271235    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-sysctl-d\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.273713 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271247    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70aa0113-0423-4bdf-8335-d6c5bd707757-tmp-dir\") pod \"node-resolver-vjp7p\" (UID: \"70aa0113-0423-4bdf-8335-d6c5bd707757\") " pod="openshift-dns/node-resolver-vjp7p"
Nov 04 02:05:23.273713 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271275    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-registration-dir\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:05:23.273713 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271351    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-sysctl-d\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.273713 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271265    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-env-overrides\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.273713 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271474    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-cni-binary-copy\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.273713 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271472    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a395e26-3eac-4728-bbad-c42eaf6da56d-system-cni-dir\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:05:23.273713 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271518    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/16c89b3c-f63f-4489-8eae-5918677e1e61-iptables-alerter-script\") pod \"iptables-alerter-qvk5m\" (UID: \"16c89b3c-f63f-4489-8eae-5918677e1e61\") " pod="openshift-network-operator/iptables-alerter-qvk5m"
Nov 04 02:05:23.273713 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271525    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-sys\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.273713 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271527    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a395e26-3eac-4728-bbad-c42eaf6da56d-system-cni-dir\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:05:23.273713 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271546    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-tuned\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.273713 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271571    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pl9r\" (UniqueName: \"kubernetes.io/projected/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-kube-api-access-9pl9r\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.273713 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271592    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-sys\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.273713 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271596    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8ddd2cf3-5cce-41a8-ad26-6946d3cb5910-serviceca\") pod \"node-ca-gzg65\" (UID: \"8ddd2cf3-5cce-41a8-ad26-6946d3cb5910\") " pod="openshift-image-registry/node-ca-gzg65"
Nov 04 02:05:23.273713 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271593    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-kubelet\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.273713 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271638    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-kubelet\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.274508 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271640    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-run-ovn-kubernetes\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.274508 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271667    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9a395e26-3eac-4728-bbad-c42eaf6da56d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:05:23.274508 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271687    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ddd2cf3-5cce-41a8-ad26-6946d3cb5910-host\") pod \"node-ca-gzg65\" (UID: \"8ddd2cf3-5cce-41a8-ad26-6946d3cb5910\") " pod="openshift-image-registry/node-ca-gzg65"
Nov 04 02:05:23.274508 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271702    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-run-netns\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.274508 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271724    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-run-openvswitch\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.274508 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271746    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-run-ovn\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.274508 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271776    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-node-log\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.274508 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271802    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-ovnkube-config\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.274508 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271805    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-run-netns\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.274508 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271826    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkch2\" (UniqueName: \"kubernetes.io/projected/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-kube-api-access-hkch2\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.274508 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271835    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ddd2cf3-5cce-41a8-ad26-6946d3cb5910-host\") pod \"node-ca-gzg65\" (UID: \"8ddd2cf3-5cce-41a8-ad26-6946d3cb5910\") " pod="openshift-image-registry/node-ca-gzg65"
Nov 04 02:05:23.274508 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271857    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9a395e26-3eac-4728-bbad-c42eaf6da56d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:05:23.274508 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271877    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-node-log\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.274508 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271883    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-hostroot\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.274508 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271907    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-cni-bin\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.274508 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271910    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-run-openvswitch\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.274508 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271931    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr84q\" (UniqueName: \"kubernetes.io/projected/afe4b3d8-a18c-4587-9057-83123ffe3347-kube-api-access-fr84q\") pod \"network-metrics-daemon-4v7xm\" (UID: \"afe4b3d8-a18c-4587-9057-83123ffe3347\") " pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:05:23.275259 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271943    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-run-ovn\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.275259 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271954    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-sys-fs\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:05:23.275259 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271974    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-run-ovn-kubernetes\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.275259 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.271978    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-slash\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.275259 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.272004    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-run\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.275259 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.272017    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-slash\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.275259 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.272024    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a395e26-3eac-4728-bbad-c42eaf6da56d-cni-binary-copy\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:05:23.275259 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.272054    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-cnibin\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.275259 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.272067    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-run\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.275259 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.272081    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-multus-daemon-config\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.275259 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.272126    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-482t2\" (UniqueName: \"kubernetes.io/projected/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-kube-api-access-482t2\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:05:23.275259 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.272134    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-cni-bin\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.275259 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.272151    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/39235bb9-36b4-43e4-9161-043a3fb6a261-agent-certs\") pod \"konnectivity-agent-928nm\" (UID: \"39235bb9-36b4-43e4-9161-043a3fb6a261\") " pod="kube-system/konnectivity-agent-928nm"
Nov 04 02:05:23.275259 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.272175    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-kubernetes\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.275259 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.272199    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-run-systemd\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.275259 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.272270    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-run-systemd\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.275259 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.272384    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9a395e26-3eac-4728-bbad-c42eaf6da56d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:05:23.275757 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.272447    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a395e26-3eac-4728-bbad-c42eaf6da56d-cni-binary-copy\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:05:23.275757 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.272464    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-cnibin\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.275757 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.272536    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-sys-fs\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:05:23.275757 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.272700    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-ovnkube-script-lib\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.275757 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.273075    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-multus-daemon-config\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.275757 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.272019    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/39235bb9-36b4-43e4-9161-043a3fb6a261-konnectivity-ca\") pod \"konnectivity-agent-928nm\" (UID: \"39235bb9-36b4-43e4-9161-043a3fb6a261\") " pod="kube-system/konnectivity-agent-928nm"
Nov 04 02:05:23.275757 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.273147    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-env-overrides\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.275757 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.273169    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-hostroot\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.275757 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.273212    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-kubernetes\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.275757 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.273670    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-ovnkube-config\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.275757 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.274189    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9a395e26-3eac-4728-bbad-c42eaf6da56d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:05:23.275757 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.274917    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-tmp\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.275757 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.275076    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-tuned\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.275757 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.275197    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-ovn-node-metrics-cert\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.275757 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.275335    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/39235bb9-36b4-43e4-9161-043a3fb6a261-agent-certs\") pod \"konnectivity-agent-928nm\" (UID: \"39235bb9-36b4-43e4-9161-043a3fb6a261\") " pod="kube-system/konnectivity-agent-928nm"
Nov 04 02:05:23.329824 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:23.329803    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 02:05:23.329950 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:23.329830    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 02:05:23.329950 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:23.329842    2582 projected.go:196] Error preparing data for projected volume kube-api-access-nrkqj for pod openshift-network-diagnostics/network-check-target-98h6z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 02:05:23.329950 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:23.329889    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b0a72217-fe1d-4589-9201-651503694cb4-kube-api-access-nrkqj podName:b0a72217-fe1d-4589-9201-651503694cb4 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:23.82987494 +0000 UTC m=+3.307962712 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-nrkqj" (UniqueName: "kubernetes.io/projected/b0a72217-fe1d-4589-9201-651503694cb4-kube-api-access-nrkqj") pod "network-check-target-98h6z" (UID: "b0a72217-fe1d-4589-9201-651503694cb4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 02:05:23.332221 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.332195    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcv2t\" (UniqueName: \"kubernetes.io/projected/70aa0113-0423-4bdf-8335-d6c5bd707757-kube-api-access-fcv2t\") pod \"node-resolver-vjp7p\" (UID: \"70aa0113-0423-4bdf-8335-d6c5bd707757\") " pod="openshift-dns/node-resolver-vjp7p"
Nov 04 02:05:23.332315 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.332273    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcxzt\" (UniqueName: \"kubernetes.io/projected/8ddd2cf3-5cce-41a8-ad26-6946d3cb5910-kube-api-access-lcxzt\") pod \"node-ca-gzg65\" (UID: \"8ddd2cf3-5cce-41a8-ad26-6946d3cb5910\") " pod="openshift-image-registry/node-ca-gzg65"
Nov 04 02:05:23.332422 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.332404    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l89wj\" (UniqueName: \"kubernetes.io/projected/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-kube-api-access-l89wj\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.332473 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.332440    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85d9k\" (UniqueName: \"kubernetes.io/projected/16c89b3c-f63f-4489-8eae-5918677e1e61-kube-api-access-85d9k\") pod \"iptables-alerter-qvk5m\" (UID: \"16c89b3c-f63f-4489-8eae-5918677e1e61\") " pod="openshift-network-operator/iptables-alerter-qvk5m"
Nov 04 02:05:23.332623 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.332604    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84hdg\" (UniqueName: \"kubernetes.io/projected/9a395e26-3eac-4728-bbad-c42eaf6da56d-kube-api-access-84hdg\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:05:23.334029 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.334009    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkch2\" (UniqueName: \"kubernetes.io/projected/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-kube-api-access-hkch2\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.334273 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.334254    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pl9r\" (UniqueName: \"kubernetes.io/projected/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-kube-api-access-9pl9r\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.334310 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.334273    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-482t2\" (UniqueName: \"kubernetes.io/projected/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-kube-api-access-482t2\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:05:23.334349 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.334332    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr84q\" (UniqueName: \"kubernetes.io/projected/afe4b3d8-a18c-4587-9057-83123ffe3347-kube-api-access-fr84q\") pod \"network-metrics-daemon-4v7xm\" (UID: \"afe4b3d8-a18c-4587-9057-83123ffe3347\") " pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:05:23.432951 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.432931    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vjp7p"
Nov 04 02:05:23.438083 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.438064    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gzg65"
Nov 04 02:05:23.442945 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:23.442921    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70aa0113_0423_4bdf_8335_d6c5bd707757.slice/crio-67c2a63861bcb1cccf0e3cf5e68e34ba2820092939e28368daf4581454e811fc WatchSource:0}: Error finding container 67c2a63861bcb1cccf0e3cf5e68e34ba2820092939e28368daf4581454e811fc: Status 404 returned error can't find the container with id 67c2a63861bcb1cccf0e3cf5e68e34ba2820092939e28368daf4581454e811fc
Nov 04 02:05:23.446335 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.446311    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qvk5m"
Nov 04 02:05:23.447408 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:23.447389    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ddd2cf3_5cce_41a8_ad26_6946d3cb5910.slice/crio-d98fb4942db8640f591aad36e7dcd313c88fb76f0789cb58851ffeaf8aa1e0db WatchSource:0}: Error finding container d98fb4942db8640f591aad36e7dcd313c88fb76f0789cb58851ffeaf8aa1e0db: Status 404 returned error can't find the container with id d98fb4942db8640f591aad36e7dcd313c88fb76f0789cb58851ffeaf8aa1e0db
Nov 04 02:05:23.453949 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.453933    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:05:23.456105 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.456077    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:05:23.456511 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:23.456491    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16c89b3c_f63f_4489_8eae_5918677e1e61.slice/crio-951057a68760a213b97aa13d3711a9ed29ed8a6657aba282957df5e18f087b24 WatchSource:0}: Error finding container 951057a68760a213b97aa13d3711a9ed29ed8a6657aba282957df5e18f087b24: Status 404 returned error can't find the container with id 951057a68760a213b97aa13d3711a9ed29ed8a6657aba282957df5e18f087b24
Nov 04 02:05:23.463286 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:23.463265    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94fedaf4_9e63_4c96_bb3e_e508815dd6c2.slice/crio-051003e6c47b203cbf6bcdec1d60dbc6c8426203c1020ef1924f9236b7b7acd8 WatchSource:0}: Error finding container 051003e6c47b203cbf6bcdec1d60dbc6c8426203c1020ef1924f9236b7b7acd8: Status 404 returned error can't find the container with id 051003e6c47b203cbf6bcdec1d60dbc6c8426203c1020ef1924f9236b7b7acd8
Nov 04 02:05:23.465633 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:23.465616    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod078e0741_51f6_4ae3_b9c0_bdf82a0f8f93.slice/crio-d0a5ffa2c44728ef3ea5e4942d323bbb760ae3c3d464e28cc53c8ea1f9827a2a WatchSource:0}: Error finding container d0a5ffa2c44728ef3ea5e4942d323bbb760ae3c3d464e28cc53c8ea1f9827a2a: Status 404 returned error can't find the container with id d0a5ffa2c44728ef3ea5e4942d323bbb760ae3c3d464e28cc53c8ea1f9827a2a
Nov 04 02:05:23.471160 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.471142    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-928nm"
Nov 04 02:05:23.481614 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:23.481595    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39235bb9_36b4_43e4_9161_043a3fb6a261.slice/crio-e22425b440bc67c66afa0eebbedb50069edb060ce7965835cdab6c3da015f5a1 WatchSource:0}: Error finding container e22425b440bc67c66afa0eebbedb50069edb060ce7965835cdab6c3da015f5a1: Status 404 returned error can't find the container with id e22425b440bc67c66afa0eebbedb50069edb060ce7965835cdab6c3da015f5a1
Nov 04 02:05:23.482929 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.482868    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hclbr"
Nov 04 02:05:23.484993 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.484972    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:23.492711 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.492688    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:05:23.497399 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:23.497375    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a90ff43_d7f2_492d_98c9_31d9f6ca4b4d.slice/crio-7261e31a7f50bb3a3dde6c082a7623aea0790369aeb5aa0713fec06d6ac30689 WatchSource:0}: Error finding container 7261e31a7f50bb3a3dde6c082a7623aea0790369aeb5aa0713fec06d6ac30689: Status 404 returned error can't find the container with id 7261e31a7f50bb3a3dde6c082a7623aea0790369aeb5aa0713fec06d6ac30689
Nov 04 02:05:23.500564 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:23.500549    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7f9b7e3_b61b_4756_a01e_f5f026c5a3e1.slice/crio-44d76dd4951701fb6150006d4c061465fed28cc004ba46595fd5bdc9c0fb21bb WatchSource:0}: Error finding container 44d76dd4951701fb6150006d4c061465fed28cc004ba46595fd5bdc9c0fb21bb: Status 404 returned error can't find the container with id 44d76dd4951701fb6150006d4c061465fed28cc004ba46595fd5bdc9c0fb21bb
Nov 04 02:05:23.503471 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:23.503452    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a395e26_3eac_4728_bbad_c42eaf6da56d.slice/crio-2764f36f82f901a7bbe63937dbc8c81acb2694d962f903f1cee7c4450283600b WatchSource:0}: Error finding container 2764f36f82f901a7bbe63937dbc8c81acb2694d962f903f1cee7c4450283600b: Status 404 returned error can't find the container with id 2764f36f82f901a7bbe63937dbc8c81acb2694d962f903f1cee7c4450283600b
Nov 04 02:05:23.777055 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.777019    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e164905d-5505-4d20-af2e-b8acd370bf18-original-pull-secret\") pod \"global-pull-secret-syncer-b4wjr\" (UID: \"e164905d-5505-4d20-af2e-b8acd370bf18\") " pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:23.777217 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.777110    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs\") pod \"network-metrics-daemon-4v7xm\" (UID: \"afe4b3d8-a18c-4587-9057-83123ffe3347\") " pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:05:23.777217 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:23.777177    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 02:05:23.777217 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:23.777207    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 02:05:23.777337 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:23.777238    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e164905d-5505-4d20-af2e-b8acd370bf18-original-pull-secret podName:e164905d-5505-4d20-af2e-b8acd370bf18 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:24.777218561 +0000 UTC m=+4.255306315 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e164905d-5505-4d20-af2e-b8acd370bf18-original-pull-secret") pod "global-pull-secret-syncer-b4wjr" (UID: "e164905d-5505-4d20-af2e-b8acd370bf18") : object "kube-system"/"original-pull-secret" not registered
Nov 04 02:05:23.777337 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:23.777256    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs podName:afe4b3d8-a18c-4587-9057-83123ffe3347 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:24.777248679 +0000 UTC m=+4.255336431 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs") pod "network-metrics-daemon-4v7xm" (UID: "afe4b3d8-a18c-4587-9057-83123ffe3347") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 02:05:23.845206 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.845180    2582 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 02:05:23.877624 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.877597    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrkqj\" (UniqueName: \"kubernetes.io/projected/b0a72217-fe1d-4589-9201-651503694cb4-kube-api-access-nrkqj\") pod \"network-check-target-98h6z\" (UID: \"b0a72217-fe1d-4589-9201-651503694cb4\") " pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:05:23.877753 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:23.877739    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 02:05:23.877817 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:23.877758    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 02:05:23.877817 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:23.877770    2582 projected.go:196] Error preparing data for projected volume kube-api-access-nrkqj for pod openshift-network-diagnostics/network-check-target-98h6z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 02:05:23.877900 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:23.877830    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b0a72217-fe1d-4589-9201-651503694cb4-kube-api-access-nrkqj podName:b0a72217-fe1d-4589-9201-651503694cb4 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:24.877812477 +0000 UTC m=+4.355900242 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-nrkqj" (UniqueName: "kubernetes.io/projected/b0a72217-fe1d-4589-9201-651503694cb4-kube-api-access-nrkqj") pod "network-check-target-98h6z" (UID: "b0a72217-fe1d-4589-9201-651503694cb4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 02:05:23.988786 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.988752    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-98zkj" event={"ID":"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93","Type":"ContainerStarted","Data":"d0a5ffa2c44728ef3ea5e4942d323bbb760ae3c3d464e28cc53c8ea1f9827a2a"}
Nov 04 02:05:23.989850 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.989824    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qvk5m" event={"ID":"16c89b3c-f63f-4489-8eae-5918677e1e61","Type":"ContainerStarted","Data":"951057a68760a213b97aa13d3711a9ed29ed8a6657aba282957df5e18f087b24"}
Nov 04 02:05:23.990889 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.990871    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hclbr" event={"ID":"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d","Type":"ContainerStarted","Data":"7261e31a7f50bb3a3dde6c082a7623aea0790369aeb5aa0713fec06d6ac30689"}
Nov 04 02:05:23.992015 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.991995    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb" event={"ID":"94fedaf4-9e63-4c96-bb3e-e508815dd6c2","Type":"ContainerStarted","Data":"051003e6c47b203cbf6bcdec1d60dbc6c8426203c1020ef1924f9236b7b7acd8"}
Nov 04 02:05:23.993009 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.992991    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gzg65" event={"ID":"8ddd2cf3-5cce-41a8-ad26-6946d3cb5910","Type":"ContainerStarted","Data":"d98fb4942db8640f591aad36e7dcd313c88fb76f0789cb58851ffeaf8aa1e0db"}
Nov 04 02:05:23.994044 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.994026    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vjp7p" event={"ID":"70aa0113-0423-4bdf-8335-d6c5bd707757","Type":"ContainerStarted","Data":"67c2a63861bcb1cccf0e3cf5e68e34ba2820092939e28368daf4581454e811fc"}
Nov 04 02:05:23.995077 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.995060    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jsh5m" event={"ID":"9a395e26-3eac-4728-bbad-c42eaf6da56d","Type":"ContainerStarted","Data":"2764f36f82f901a7bbe63937dbc8c81acb2694d962f903f1cee7c4450283600b"}
Nov 04 02:05:23.996053 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.996035    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2l29t" event={"ID":"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1","Type":"ContainerStarted","Data":"44d76dd4951701fb6150006d4c061465fed28cc004ba46595fd5bdc9c0fb21bb"}
Nov 04 02:05:23.997060 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:23.997044    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-928nm" event={"ID":"39235bb9-36b4-43e4-9161-043a3fb6a261","Type":"ContainerStarted","Data":"e22425b440bc67c66afa0eebbedb50069edb060ce7965835cdab6c3da015f5a1"}
Nov 04 02:05:24.234854 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:24.234765    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 02:00:22 +0000 UTC" deadline="2028-05-02 16:12:03.391176135 +0000 UTC"
Nov 04 02:05:24.234854 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:24.234830    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13094h6m39.156372914s"
Nov 04 02:05:24.784896 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:24.784860    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs\") pod \"network-metrics-daemon-4v7xm\" (UID: \"afe4b3d8-a18c-4587-9057-83123ffe3347\") " pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:05:24.785072 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:24.784933    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e164905d-5505-4d20-af2e-b8acd370bf18-original-pull-secret\") pod \"global-pull-secret-syncer-b4wjr\" (UID: \"e164905d-5505-4d20-af2e-b8acd370bf18\") " pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:24.785072 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:24.785066    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 02:05:24.785216 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:24.785136    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e164905d-5505-4d20-af2e-b8acd370bf18-original-pull-secret podName:e164905d-5505-4d20-af2e-b8acd370bf18 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:26.785119322 +0000 UTC m=+6.263207088 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e164905d-5505-4d20-af2e-b8acd370bf18-original-pull-secret") pod "global-pull-secret-syncer-b4wjr" (UID: "e164905d-5505-4d20-af2e-b8acd370bf18") : object "kube-system"/"original-pull-secret" not registered
Nov 04 02:05:24.785216 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:24.785202    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 02:05:24.785328 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:24.785235    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs podName:afe4b3d8-a18c-4587-9057-83123ffe3347 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:26.785224005 +0000 UTC m=+6.263311774 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs") pod "network-metrics-daemon-4v7xm" (UID: "afe4b3d8-a18c-4587-9057-83123ffe3347") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 02:05:24.886058 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:24.886002    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrkqj\" (UniqueName: \"kubernetes.io/projected/b0a72217-fe1d-4589-9201-651503694cb4-kube-api-access-nrkqj\") pod \"network-check-target-98h6z\" (UID: \"b0a72217-fe1d-4589-9201-651503694cb4\") " pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:05:24.886656 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:24.886196    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 02:05:24.886656 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:24.886215    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 02:05:24.886656 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:24.886229    2582 projected.go:196] Error preparing data for projected volume kube-api-access-nrkqj for pod openshift-network-diagnostics/network-check-target-98h6z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 02:05:24.886656 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:24.886285    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b0a72217-fe1d-4589-9201-651503694cb4-kube-api-access-nrkqj podName:b0a72217-fe1d-4589-9201-651503694cb4 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:26.886266315 +0000 UTC m=+6.364354083 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-nrkqj" (UniqueName: "kubernetes.io/projected/b0a72217-fe1d-4589-9201-651503694cb4-kube-api-access-nrkqj") pod "network-check-target-98h6z" (UID: "b0a72217-fe1d-4589-9201-651503694cb4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 02:05:24.982240 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:24.982213    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:05:24.982392 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:24.982329    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98h6z" podUID="b0a72217-fe1d-4589-9201-651503694cb4"
Nov 04 02:05:24.982449 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:24.982414    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:05:24.982516 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:24.982500    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v7xm" podUID="afe4b3d8-a18c-4587-9057-83123ffe3347"
Nov 04 02:05:24.987549 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:24.987310    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:24.987549 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:24.987482    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4wjr" podUID="e164905d-5505-4d20-af2e-b8acd370bf18"
Nov 04 02:05:25.235338 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:25.235216    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 02:00:22 +0000 UTC" deadline="2028-07-17 17:52:43.338103266 +0000 UTC"
Nov 04 02:05:25.235338 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:25.235274    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14919h47m18.102832781s"
Nov 04 02:05:25.387441 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:25.387414    2582 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 02:05:25.421302 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:25.421277    2582 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 02:05:26.802607 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:26.802368    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e164905d-5505-4d20-af2e-b8acd370bf18-original-pull-secret\") pod \"global-pull-secret-syncer-b4wjr\" (UID: \"e164905d-5505-4d20-af2e-b8acd370bf18\") " pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:26.803027 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:26.802665    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs\") pod \"network-metrics-daemon-4v7xm\" (UID: \"afe4b3d8-a18c-4587-9057-83123ffe3347\") " pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:05:26.803027 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:26.802788    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 02:05:26.803027 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:26.802806    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 02:05:26.803027 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:26.802850    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs podName:afe4b3d8-a18c-4587-9057-83123ffe3347 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:30.80283214 +0000 UTC m=+10.280919894 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs") pod "network-metrics-daemon-4v7xm" (UID: "afe4b3d8-a18c-4587-9057-83123ffe3347") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 02:05:26.803027 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:26.802870    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e164905d-5505-4d20-af2e-b8acd370bf18-original-pull-secret podName:e164905d-5505-4d20-af2e-b8acd370bf18 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:30.802859978 +0000 UTC m=+10.280947736 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e164905d-5505-4d20-af2e-b8acd370bf18-original-pull-secret") pod "global-pull-secret-syncer-b4wjr" (UID: "e164905d-5505-4d20-af2e-b8acd370bf18") : object "kube-system"/"original-pull-secret" not registered
Nov 04 02:05:26.903937 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:26.903906    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrkqj\" (UniqueName: \"kubernetes.io/projected/b0a72217-fe1d-4589-9201-651503694cb4-kube-api-access-nrkqj\") pod \"network-check-target-98h6z\" (UID: \"b0a72217-fe1d-4589-9201-651503694cb4\") " pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:05:26.904114 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:26.904077    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 02:05:26.904114 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:26.904115    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 02:05:26.904226 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:26.904127    2582 projected.go:196] Error preparing data for projected volume kube-api-access-nrkqj for pod openshift-network-diagnostics/network-check-target-98h6z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 02:05:26.904226 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:26.904176    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b0a72217-fe1d-4589-9201-651503694cb4-kube-api-access-nrkqj podName:b0a72217-fe1d-4589-9201-651503694cb4 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:30.904159408 +0000 UTC m=+10.382247174 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-nrkqj" (UniqueName: "kubernetes.io/projected/b0a72217-fe1d-4589-9201-651503694cb4-kube-api-access-nrkqj") pod "network-check-target-98h6z" (UID: "b0a72217-fe1d-4589-9201-651503694cb4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 02:05:26.982792 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:26.982556    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:26.982792 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:26.982621    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:05:26.982792 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:26.982729    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:05:26.982792 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:26.982739    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4wjr" podUID="e164905d-5505-4d20-af2e-b8acd370bf18"
Nov 04 02:05:26.983019 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:26.982840    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98h6z" podUID="b0a72217-fe1d-4589-9201-651503694cb4"
Nov 04 02:05:26.983019 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:26.982933    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v7xm" podUID="afe4b3d8-a18c-4587-9057-83123ffe3347"
Nov 04 02:05:27.035466 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:27.035276    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jsh5m" event={"ID":"9a395e26-3eac-4728-bbad-c42eaf6da56d","Type":"ContainerStarted","Data":"7ab82a1249d696be0410a2bf34b49dd839f561e2a7568a1b7297abfe9a0084f1"}
Nov 04 02:05:27.043410 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:27.042961    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal" event={"ID":"5dd9721b3a00168bde79d6cad3060599","Type":"ContainerStarted","Data":"a28dc15040456383218742865fd3d0a93eebbe2a0a4f754868fabb0ea9750afb"}
Nov 04 02:05:27.046009 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:27.045567    2582 generic.go:356] "Generic (PLEG): container finished" podID="1ec35563e4041daa397c9c3ffc626b1c" containerID="aa72b53adf7911d7c704e045a8bd15aa7130a7ca3660c6470d64bd4074e87ae0" exitCode=0
Nov 04 02:05:27.046009 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:27.045600    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal" event={"ID":"1ec35563e4041daa397c9c3ffc626b1c","Type":"ContainerDied","Data":"aa72b53adf7911d7c704e045a8bd15aa7130a7ca3660c6470d64bd4074e87ae0"}
Nov 04 02:05:28.052258 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:28.052227    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal" event={"ID":"1ec35563e4041daa397c9c3ffc626b1c","Type":"ContainerStarted","Data":"f1fa2effa39f93d396b21fbef0c5f5a734494d27146f8b2e4550d6b6f28c1960"}
Nov 04 02:05:28.055399 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:28.055371    2582 generic.go:356] "Generic (PLEG): container finished" podID="9a395e26-3eac-4728-bbad-c42eaf6da56d" containerID="7ab82a1249d696be0410a2bf34b49dd839f561e2a7568a1b7297abfe9a0084f1" exitCode=0
Nov 04 02:05:28.056482 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:28.056385    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jsh5m" event={"ID":"9a395e26-3eac-4728-bbad-c42eaf6da56d","Type":"ContainerDied","Data":"7ab82a1249d696be0410a2bf34b49dd839f561e2a7568a1b7297abfe9a0084f1"}
Nov 04 02:05:28.178573 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:28.178262    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal" podStartSLOduration=6.178243128 podStartE2EDuration="6.178243128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:05:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:05:28.177294783 +0000 UTC m=+7.655382559" watchObservedRunningTime="2026-11-04 02:05:28.178243128 +0000 UTC m=+7.656330904"
Nov 04 02:05:28.178573 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:28.178425    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal" podStartSLOduration=6.178416671 podStartE2EDuration="6.178416671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:05:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:05:27.514662148 +0000 UTC m=+6.992749918" watchObservedRunningTime="2026-11-04 02:05:28.178416671 +0000 UTC m=+7.656504446"
Nov 04 02:05:28.982709 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:28.982289    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:28.982709 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:28.982334    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:05:28.982709 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:28.982443    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4wjr" podUID="e164905d-5505-4d20-af2e-b8acd370bf18"
Nov 04 02:05:28.982709 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:28.982529    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98h6z" podUID="b0a72217-fe1d-4589-9201-651503694cb4"
Nov 04 02:05:28.982709 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:28.982590    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:05:28.982709 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:28.982669    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v7xm" podUID="afe4b3d8-a18c-4587-9057-83123ffe3347"
Nov 04 02:05:29.062462 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:29.062408    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-928nm" event={"ID":"39235bb9-36b4-43e4-9161-043a3fb6a261","Type":"ContainerStarted","Data":"1594486df6691eea2b747834dcff7d35b4ec2fe8d23503142736fcdff0bf0228"}
Nov 04 02:05:30.067147 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:30.067106    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb" event={"ID":"94fedaf4-9e63-4c96-bb3e-e508815dd6c2","Type":"ContainerStarted","Data":"aa0f79954b58fc4bbd1557cd4c723e35b087e9f3c90af8f8b3cbc9068bc898cc"}
Nov 04 02:05:30.070207 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:30.070175    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gzg65" event={"ID":"8ddd2cf3-5cce-41a8-ad26-6946d3cb5910","Type":"ContainerStarted","Data":"10ff5379143fa3b8184ec69c850e09ee89f482f2e8559e737b0f9175209ba30d"}
Nov 04 02:05:30.185890 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:30.185832    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-928nm" podStartSLOduration=3.774303064 podStartE2EDuration="8.185809839s" totalImagesPullingTime="4.411506775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:05:29.179297397 +0000 UTC m=+8.657385171" watchObservedRunningTime="2026-11-04 02:05:30.185809839 +0000 UTC m=+9.663897621"
Nov 04 02:05:30.837530 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:30.837493    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs\") pod \"network-metrics-daemon-4v7xm\" (UID: \"afe4b3d8-a18c-4587-9057-83123ffe3347\") " pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:05:30.837708 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:30.837622    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e164905d-5505-4d20-af2e-b8acd370bf18-original-pull-secret\") pod \"global-pull-secret-syncer-b4wjr\" (UID: \"e164905d-5505-4d20-af2e-b8acd370bf18\") " pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:30.837708 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:30.837643    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 02:05:30.837708 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:30.837708    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs podName:afe4b3d8-a18c-4587-9057-83123ffe3347 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:38.837688054 +0000 UTC m=+18.315775827 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs") pod "network-metrics-daemon-4v7xm" (UID: "afe4b3d8-a18c-4587-9057-83123ffe3347") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 02:05:30.837836 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:30.837732    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 02:05:30.837836 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:30.837782    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e164905d-5505-4d20-af2e-b8acd370bf18-original-pull-secret podName:e164905d-5505-4d20-af2e-b8acd370bf18 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:38.837767651 +0000 UTC m=+18.315855411 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e164905d-5505-4d20-af2e-b8acd370bf18-original-pull-secret") pod "global-pull-secret-syncer-b4wjr" (UID: "e164905d-5505-4d20-af2e-b8acd370bf18") : object "kube-system"/"original-pull-secret" not registered
Nov 04 02:05:30.938402 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:30.938377    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrkqj\" (UniqueName: \"kubernetes.io/projected/b0a72217-fe1d-4589-9201-651503694cb4-kube-api-access-nrkqj\") pod \"network-check-target-98h6z\" (UID: \"b0a72217-fe1d-4589-9201-651503694cb4\") " pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:05:30.938525 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:30.938510    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 02:05:30.938583 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:30.938538    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 02:05:30.938583 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:30.938551    2582 projected.go:196] Error preparing data for projected volume kube-api-access-nrkqj for pod openshift-network-diagnostics/network-check-target-98h6z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 02:05:30.938683 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:30.938599    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b0a72217-fe1d-4589-9201-651503694cb4-kube-api-access-nrkqj podName:b0a72217-fe1d-4589-9201-651503694cb4 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:38.938581837 +0000 UTC m=+18.416669612 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-nrkqj" (UniqueName: "kubernetes.io/projected/b0a72217-fe1d-4589-9201-651503694cb4-kube-api-access-nrkqj") pod "network-check-target-98h6z" (UID: "b0a72217-fe1d-4589-9201-651503694cb4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 02:05:30.986041 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:30.985420    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:05:30.986041 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:30.985566    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98h6z" podUID="b0a72217-fe1d-4589-9201-651503694cb4"
Nov 04 02:05:30.986041 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:30.985662    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:05:30.986041 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:30.985801    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v7xm" podUID="afe4b3d8-a18c-4587-9057-83123ffe3347"
Nov 04 02:05:30.986041 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:30.985874    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:30.986041 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:30.985985    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4wjr" podUID="e164905d-5505-4d20-af2e-b8acd370bf18"
Nov 04 02:05:31.278607 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:31.278514    2582 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 04 02:05:31.913758 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:31.913211    2582 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T02:05:31.278532734Z","UUID":"ffcdda72-ec65-47ed-b1e6-e9215f81f475","Handler":null,"Name":"","Endpoint":""}
Nov 04 02:05:31.914983 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:31.914621    2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 04 02:05:31.914983 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:31.914649    2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 04 02:05:32.075801 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:32.075767    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hclbr" event={"ID":"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d","Type":"ContainerStarted","Data":"a2bbb7caec1e8dffd8806e3615b36ab38e63b92c5eb080b83e7dc4be1dd81569"}
Nov 04 02:05:32.078390 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:32.078363    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb" event={"ID":"94fedaf4-9e63-4c96-bb3e-e508815dd6c2","Type":"ContainerStarted","Data":"81fd0d06333c2cbbed42ba18be1f7ed93633271dca5aeedce0781f888a346f02"}
Nov 04 02:05:32.079819 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:32.079796    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vjp7p" event={"ID":"70aa0113-0423-4bdf-8335-d6c5bd707757","Type":"ContainerStarted","Data":"96b3f24d8d8eee24c6ae0fc5ccb31173a9b94f2a3e5af8dd0233ac1c1ba953d1"}
Nov 04 02:05:32.195741 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:32.195693    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-hclbr" podStartSLOduration=2.484875789 podStartE2EDuration="10.195680878s" totalImagesPullingTime="7.710805089s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:05:32.195183993 +0000 UTC m=+11.673271764" watchObservedRunningTime="2026-11-04 02:05:32.195680878 +0000 UTC m=+11.673768651"
Nov 04 02:05:32.195976 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:32.195946    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gzg65" podStartSLOduration=4.589711784 podStartE2EDuration="10.195936105s" totalImagesPullingTime="5.606224321s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:05:30.186640689 +0000 UTC m=+9.664728464" watchObservedRunningTime="2026-11-04 02:05:32.195936105 +0000 UTC m=+11.674023906"
Nov 04 02:05:32.534645 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:32.534613    2582 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-928nm"
Nov 04 02:05:32.535325 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:32.535303    2582 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-928nm"
Nov 04 02:05:32.653477 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:32.653246    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-vjp7p" podStartSLOduration=2.409068377 podStartE2EDuration="10.653226711s" totalImagesPullingTime="8.244158334s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:05:32.312400821 +0000 UTC m=+11.790488575" watchObservedRunningTime="2026-11-04 02:05:32.653226711 +0000 UTC m=+12.131314486"
Nov 04 02:05:32.982414 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:32.982066    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:05:32.982414 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:32.982066    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:05:32.982414 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:32.982373    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v7xm" podUID="afe4b3d8-a18c-4587-9057-83123ffe3347"
Nov 04 02:05:32.982414 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:32.982082    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:32.982704 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:32.982464    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98h6z" podUID="b0a72217-fe1d-4589-9201-651503694cb4"
Nov 04 02:05:32.982704 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:32.982501    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4wjr" podUID="e164905d-5505-4d20-af2e-b8acd370bf18"
Nov 04 02:05:33.084249 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:33.084217    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qvk5m" event={"ID":"16c89b3c-f63f-4489-8eae-5918677e1e61","Type":"ContainerStarted","Data":"df8143db6960aac9667f8b0125c3939ff9f1a1fe44158a05c0279d7937f3ba28"}
Nov 04 02:05:33.086929 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:33.086899    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb" event={"ID":"94fedaf4-9e63-4c96-bb3e-e508815dd6c2","Type":"ContainerStarted","Data":"343f648f8cc7983e9de70404651039dca7f485eef3c90fe8e342a6045bff9252"}
Nov 04 02:05:33.089513 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:33.089490    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jsh5m" event={"ID":"9a395e26-3eac-4728-bbad-c42eaf6da56d","Type":"ContainerStarted","Data":"7c47f05c740a168a51556d758528a767ca77c1618027af7dc423a98707963ae6"}
Nov 04 02:05:33.090341 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:33.090320    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-928nm"
Nov 04 02:05:33.092445 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:33.092426    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-928nm"
Nov 04 02:05:33.321516 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:33.321469    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qvk5m" podStartSLOduration=3.843842216 podStartE2EDuration="11.321451705s" totalImagesPullingTime="7.477609489s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:05:33.201337356 +0000 UTC m=+12.679425133" watchObservedRunningTime="2026-11-04 02:05:33.321451705 +0000 UTC m=+12.799539480"
Nov 04 02:05:33.321694 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:33.321567    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb" podStartSLOduration=2.218323619 podStartE2EDuration="11.321562358s" totalImagesPullingTime="9.103238739s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:05:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:05:33.320876393 +0000 UTC m=+12.798964171" watchObservedRunningTime="2026-11-04 02:05:33.321562358 +0000 UTC m=+12.799650132"
Nov 04 02:05:34.094232 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:34.094156    2582 generic.go:356] "Generic (PLEG): container finished" podID="9a395e26-3eac-4728-bbad-c42eaf6da56d" containerID="7c47f05c740a168a51556d758528a767ca77c1618027af7dc423a98707963ae6" exitCode=0
Nov 04 02:05:34.094733 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:34.094231    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jsh5m" event={"ID":"9a395e26-3eac-4728-bbad-c42eaf6da56d","Type":"ContainerDied","Data":"7c47f05c740a168a51556d758528a767ca77c1618027af7dc423a98707963ae6"}
Nov 04 02:05:34.982347 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:34.982317    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:34.982520 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:34.982381    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:05:34.982520 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:34.982377    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:05:34.982520 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:34.982500    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4wjr" podUID="e164905d-5505-4d20-af2e-b8acd370bf18"
Nov 04 02:05:34.982663 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:34.982616    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98h6z" podUID="b0a72217-fe1d-4589-9201-651503694cb4"
Nov 04 02:05:34.982727 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:34.982692    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v7xm" podUID="afe4b3d8-a18c-4587-9057-83123ffe3347"
Nov 04 02:05:35.097845 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:35.097817    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-98zkj" event={"ID":"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93","Type":"ContainerStarted","Data":"da0bfc363a7a8ece84818a114a171d620e328281487ed982886b9a4dda32cb7c"}
Nov 04 02:05:35.217200 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:35.217144    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-98zkj" podStartSLOduration=1.6687505489999999 podStartE2EDuration="13.217130449s" totalImagesPullingTime="11.5483799s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:05:35.216767344 +0000 UTC m=+14.694855117" watchObservedRunningTime="2026-11-04 02:05:35.217130449 +0000 UTC m=+14.695218222"
Nov 04 02:05:36.102464 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:36.102431    2582 generic.go:356] "Generic (PLEG): container finished" podID="9a395e26-3eac-4728-bbad-c42eaf6da56d" containerID="a88fa3c2d03d6f89de2a0adb1c815dce46b3a160a6266127d10ea7705216c756" exitCode=0
Nov 04 02:05:36.102943 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:36.102523    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jsh5m" event={"ID":"9a395e26-3eac-4728-bbad-c42eaf6da56d","Type":"ContainerDied","Data":"a88fa3c2d03d6f89de2a0adb1c815dce46b3a160a6266127d10ea7705216c756"}
Nov 04 02:05:36.981669 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:36.981434    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:05:36.981843 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:36.981441    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:05:36.981843 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:36.981773    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98h6z" podUID="b0a72217-fe1d-4589-9201-651503694cb4"
Nov 04 02:05:36.981843 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:36.981464    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:36.981992 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:36.981857    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v7xm" podUID="afe4b3d8-a18c-4587-9057-83123ffe3347"
Nov 04 02:05:36.981992 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:36.981941    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4wjr" podUID="e164905d-5505-4d20-af2e-b8acd370bf18"
Nov 04 02:05:38.108699 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:38.108670    2582 generic.go:356] "Generic (PLEG): container finished" podID="9a395e26-3eac-4728-bbad-c42eaf6da56d" containerID="98b80cdcbc81b9d6ecda8c882280361794cc76236bcdc7ba5e6e8909527abeee" exitCode=0
Nov 04 02:05:38.109033 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:38.108709    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jsh5m" event={"ID":"9a395e26-3eac-4728-bbad-c42eaf6da56d","Type":"ContainerDied","Data":"98b80cdcbc81b9d6ecda8c882280361794cc76236bcdc7ba5e6e8909527abeee"}
Nov 04 02:05:38.899593 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:38.899427    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs\") pod \"network-metrics-daemon-4v7xm\" (UID: \"afe4b3d8-a18c-4587-9057-83123ffe3347\") " pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:05:38.899743 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:38.899623    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e164905d-5505-4d20-af2e-b8acd370bf18-original-pull-secret\") pod \"global-pull-secret-syncer-b4wjr\" (UID: \"e164905d-5505-4d20-af2e-b8acd370bf18\") " pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:38.899743 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:38.899562    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 02:05:38.899743 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:38.899706    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs podName:afe4b3d8-a18c-4587-9057-83123ffe3347 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:54.899692349 +0000 UTC m=+34.377780119 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs") pod "network-metrics-daemon-4v7xm" (UID: "afe4b3d8-a18c-4587-9057-83123ffe3347") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 02:05:38.899743 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:38.899710    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 02:05:38.899743 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:38.899745    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e164905d-5505-4d20-af2e-b8acd370bf18-original-pull-secret podName:e164905d-5505-4d20-af2e-b8acd370bf18 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:54.899734292 +0000 UTC m=+34.377822044 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e164905d-5505-4d20-af2e-b8acd370bf18-original-pull-secret") pod "global-pull-secret-syncer-b4wjr" (UID: "e164905d-5505-4d20-af2e-b8acd370bf18") : object "kube-system"/"original-pull-secret" not registered
Nov 04 02:05:38.982131 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:38.982050    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:05:38.982131 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:38.982077    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:38.982315 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:38.982155    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98h6z" podUID="b0a72217-fe1d-4589-9201-651503694cb4"
Nov 04 02:05:38.982315 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:38.982200    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:05:38.982423 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:38.982317    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v7xm" podUID="afe4b3d8-a18c-4587-9057-83123ffe3347"
Nov 04 02:05:38.982423 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:38.982410    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4wjr" podUID="e164905d-5505-4d20-af2e-b8acd370bf18"
Nov 04 02:05:39.000453 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:39.000429    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrkqj\" (UniqueName: \"kubernetes.io/projected/b0a72217-fe1d-4589-9201-651503694cb4-kube-api-access-nrkqj\") pod \"network-check-target-98h6z\" (UID: \"b0a72217-fe1d-4589-9201-651503694cb4\") " pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:05:39.000552 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:39.000534    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 02:05:39.000552 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:39.000546    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 02:05:39.000625 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:39.000554    2582 projected.go:196] Error preparing data for projected volume kube-api-access-nrkqj for pod openshift-network-diagnostics/network-check-target-98h6z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 02:05:39.000625 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:39.000590    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b0a72217-fe1d-4589-9201-651503694cb4-kube-api-access-nrkqj podName:b0a72217-fe1d-4589-9201-651503694cb4 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:55.000579011 +0000 UTC m=+34.478666776 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-nrkqj" (UniqueName: "kubernetes.io/projected/b0a72217-fe1d-4589-9201-651503694cb4-kube-api-access-nrkqj") pod "network-check-target-98h6z" (UID: "b0a72217-fe1d-4589-9201-651503694cb4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 02:05:39.113647 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:39.113621    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2l29t" event={"ID":"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1","Type":"ContainerStarted","Data":"670191f17d7177cf82bd3d3f4e0975e8b12b6545c2f90d706c4f0914da7377e2"}
Nov 04 02:05:39.114013 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:39.113691    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2l29t" event={"ID":"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1","Type":"ContainerStarted","Data":"1970e80400a96055f6c11d6ea8216a3aaf149a303a142cba9dba120dcd761c58"}
Nov 04 02:05:39.114013 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:39.113748    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2l29t" event={"ID":"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1","Type":"ContainerStarted","Data":"a9e3868537e28959de50c0188d3286d2f581047462f791e7f82b0752f84217e0"}
Nov 04 02:05:39.114013 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:39.113800    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2l29t" event={"ID":"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1","Type":"ContainerStarted","Data":"c41e9a1a38ee6c4e02f1a696f9c06cc458f8a03f86e0bb9fae9da6605cfad7f4"}
Nov 04 02:05:39.114013 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:39.113844    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2l29t" event={"ID":"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1","Type":"ContainerStarted","Data":"fe7e3fd21b100d3b9f65234d6d1ab415089de2720422e9d80687ccb8ae7a5d2f"}
Nov 04 02:05:39.114013 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:39.113889    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2l29t" event={"ID":"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1","Type":"ContainerStarted","Data":"2a4343b9ac4dc41b2ae1cbf3ef573d95d57c5cc495ec1b2df06e97e6f625c465"}
Nov 04 02:05:40.982721 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:40.982692    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:40.983068 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:40.982764    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4wjr" podUID="e164905d-5505-4d20-af2e-b8acd370bf18"
Nov 04 02:05:40.983068 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:40.982853    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:05:40.983068 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:40.982885    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:05:40.983068 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:40.982972    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98h6z" podUID="b0a72217-fe1d-4589-9201-651503694cb4"
Nov 04 02:05:40.983068 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:40.983013    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v7xm" podUID="afe4b3d8-a18c-4587-9057-83123ffe3347"
Nov 04 02:05:41.120508 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:41.120449    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2l29t" event={"ID":"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1","Type":"ContainerStarted","Data":"ad80173813683ca91d727a94a6a4d5a6288bf4c855859caf2a0044af4a83c9ea"}
Nov 04 02:05:42.124943 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:42.124909    2582 generic.go:356] "Generic (PLEG): container finished" podID="9a395e26-3eac-4728-bbad-c42eaf6da56d" containerID="6771cfcfe3192d6b89a907f38fd0cb2de2de3ca1a3f2dba8e14e72e47a1152da" exitCode=0
Nov 04 02:05:42.125485 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:42.124962    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jsh5m" event={"ID":"9a395e26-3eac-4728-bbad-c42eaf6da56d","Type":"ContainerDied","Data":"6771cfcfe3192d6b89a907f38fd0cb2de2de3ca1a3f2dba8e14e72e47a1152da"}
Nov 04 02:05:42.981572 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:42.981538    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:05:42.981572 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:42.981567    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:42.981746 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:42.981537    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:05:42.981746 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:42.981654    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98h6z" podUID="b0a72217-fe1d-4589-9201-651503694cb4"
Nov 04 02:05:42.981746 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:42.981713    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v7xm" podUID="afe4b3d8-a18c-4587-9057-83123ffe3347"
Nov 04 02:05:42.981890 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:42.981835    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4wjr" podUID="e164905d-5505-4d20-af2e-b8acd370bf18"
Nov 04 02:05:43.129490 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:43.129401    2582 generic.go:356] "Generic (PLEG): container finished" podID="9a395e26-3eac-4728-bbad-c42eaf6da56d" containerID="24b3e751dae85c639a2e9dbbc671c48b63c8e32a5823fce96b9d539cc61d02e4" exitCode=0
Nov 04 02:05:43.130184 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:43.129493    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jsh5m" event={"ID":"9a395e26-3eac-4728-bbad-c42eaf6da56d","Type":"ContainerDied","Data":"24b3e751dae85c639a2e9dbbc671c48b63c8e32a5823fce96b9d539cc61d02e4"}
Nov 04 02:05:43.133103 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:43.133065    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2l29t" event={"ID":"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1","Type":"ContainerStarted","Data":"6b3fdb5b9574446f3370c888457e5be0b8682dbf5f101c9ad0eb2acd67050318"}
Nov 04 02:05:43.133348 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:43.133316    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:43.133460 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:43.133400    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:43.133460 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:43.133446    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:43.150654 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:43.150467    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:43.150785 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:43.150771    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:05:43.384838 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:43.384736    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2l29t" podStartSLOduration=6.470459052 podStartE2EDuration="21.384723521s" totalImagesPullingTime="14.914264469s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:05:43.384274468 +0000 UTC m=+22.862362241" watchObservedRunningTime="2026-11-04 02:05:43.384723521 +0000 UTC m=+22.862811294"
Nov 04 02:05:44.139828 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:44.139783    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jsh5m" event={"ID":"9a395e26-3eac-4728-bbad-c42eaf6da56d","Type":"ContainerStarted","Data":"fe4c370ab9d38cbabf439a9c880e8a90e3ce160ca4f1d52f025b6510f8512a63"}
Nov 04 02:05:44.364842 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:44.364793    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jsh5m" podStartSLOduration=7.886132884 podStartE2EDuration="22.364781307s" totalImagesPullingTime="14.478648423s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:05:22 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:05:44.364437842 +0000 UTC m=+23.842525616" watchObservedRunningTime="2026-11-04 02:05:44.364781307 +0000 UTC m=+23.842869081"
Nov 04 02:05:44.981606 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:44.981570    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:05:44.981606 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:44.981592    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:05:44.981914 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:44.981668    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:44.981914 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:44.981784    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v7xm" podUID="afe4b3d8-a18c-4587-9057-83123ffe3347"
Nov 04 02:05:44.981914 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:44.981903    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98h6z" podUID="b0a72217-fe1d-4589-9201-651503694cb4"
Nov 04 02:05:44.982067 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:44.982031    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4wjr" podUID="e164905d-5505-4d20-af2e-b8acd370bf18"
Nov 04 02:05:45.599850 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:45.599822    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-b4wjr"]
Nov 04 02:05:45.600214 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:45.599915    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:45.600214 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:45.599994    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4wjr" podUID="e164905d-5505-4d20-af2e-b8acd370bf18"
Nov 04 02:05:45.602483 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:45.602460    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-98h6z"]
Nov 04 02:05:45.602581 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:45.602532    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:05:45.602619 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:45.602594    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98h6z" podUID="b0a72217-fe1d-4589-9201-651503694cb4"
Nov 04 02:05:45.605404 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:45.605383    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4v7xm"]
Nov 04 02:05:45.605505 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:45.605464    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:05:45.605576 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:45.605556    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v7xm" podUID="afe4b3d8-a18c-4587-9057-83123ffe3347"
Nov 04 02:05:46.982250 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:46.982222    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:46.982646 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:46.982327    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4wjr" podUID="e164905d-5505-4d20-af2e-b8acd370bf18"
Nov 04 02:05:46.982646 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:46.982377    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:05:46.982646 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:46.982463    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98h6z" podUID="b0a72217-fe1d-4589-9201-651503694cb4"
Nov 04 02:05:47.982008 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:47.981933    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:05:47.982176 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:47.982068    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v7xm" podUID="afe4b3d8-a18c-4587-9057-83123ffe3347"
Nov 04 02:05:48.981859 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:48.981822    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:48.982403 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:48.981954    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4wjr" podUID="e164905d-5505-4d20-af2e-b8acd370bf18"
Nov 04 02:05:48.982403 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:48.982002    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:05:48.982403 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:48.982109    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98h6z" podUID="b0a72217-fe1d-4589-9201-651503694cb4"
Nov 04 02:05:49.981819 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:49.981742    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:05:49.981996 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:49.981857    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v7xm" podUID="afe4b3d8-a18c-4587-9057-83123ffe3347"
Nov 04 02:05:50.982426 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:50.982395    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:50.982796 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:50.982495    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b4wjr" podUID="e164905d-5505-4d20-af2e-b8acd370bf18"
Nov 04 02:05:50.982796 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:50.982549    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:05:50.982796 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:50.982619    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98h6z" podUID="b0a72217-fe1d-4589-9201-651503694cb4"
Nov 04 02:05:51.982244 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:51.982072    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:05:51.982350 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:51.982326    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4v7xm" podUID="afe4b3d8-a18c-4587-9057-83123ffe3347"
Nov 04 02:05:52.364991 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.364959    2582 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeReady"
Nov 04 02:05:52.365469 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.365077    2582 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 04 02:05:52.482888 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.482862    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-2mklg"]
Nov 04 02:05:52.546958 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.546929    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-pn5gm"]
Nov 04 02:05:52.547213 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.547116    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-2mklg"
Nov 04 02:05:52.561439 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.561421    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-d27bw"]
Nov 04 02:05:52.561551 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.561537    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-pn5gm"
Nov 04 02:05:52.575456 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.575429    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg"]
Nov 04 02:05:52.575570 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.575551    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-d27bw"
Nov 04 02:05:52.593551 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.593524    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6595f8bf85-dmj26"]
Nov 04 02:05:52.593644 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.593630    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg"
Nov 04 02:05:52.598688 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.598670    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6l5g\" (UniqueName: \"kubernetes.io/projected/abdfaf94-b721-4ed4-8671-416e694df5bb-kube-api-access-l6l5g\") pod \"kas-connection-checker-685895c7c9-2mklg\" (UID: \"abdfaf94-b721-4ed4-8671-416e694df5bb\") " pod="kube-system/kas-connection-checker-685895c7c9-2mklg"
Nov 04 02:05:52.615192 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.615140    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xxtnw"]
Nov 04 02:05:52.615302 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.615282    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:05:52.635352 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.635333    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Nov 04 02:05:52.635463 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.635356    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Nov 04 02:05:52.639119 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.639076    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-6cc2t"]
Nov 04 02:05:52.642051 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.642031    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xxtnw"
Nov 04 02:05:52.652535 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.652519    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Nov 04 02:05:52.661032 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.661017    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5bd9d74c6b-thtzz"]
Nov 04 02:05:52.661145 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.661131    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:05:52.662511 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.662496    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-lb5t8\""
Nov 04 02:05:52.689179 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.689160    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-96g6r"]
Nov 04 02:05:52.689248 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.689215    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:05:52.699222 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.699206    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-stats-auth\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:05:52.699301 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.699234    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqvtx\" (UniqueName: \"kubernetes.io/projected/1597f8a8-5b13-4076-9d95-c494cf3ad5dc-kube-api-access-nqvtx\") pod \"kas-connection-checker-685895c7c9-d27bw\" (UID: \"1597f8a8-5b13-4076-9d95-c494cf3ad5dc\") " pod="kube-system/kas-connection-checker-685895c7c9-d27bw"
Nov 04 02:05:52.699301 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.699254    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-btjpg\" (UID: \"63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg"
Nov 04 02:05:52.699387 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.699317    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6l5g\" (UniqueName: \"kubernetes.io/projected/abdfaf94-b721-4ed4-8671-416e694df5bb-kube-api-access-l6l5g\") pod \"kas-connection-checker-685895c7c9-2mklg\" (UID: \"abdfaf94-b721-4ed4-8671-416e694df5bb\") " pod="kube-system/kas-connection-checker-685895c7c9-2mklg"
Nov 04 02:05:52.699387 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.699354    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-btjpg\" (UID: \"63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg"
Nov 04 02:05:52.699485 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.699436    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:05:52.699537 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.699509    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vb2x\" (UniqueName: \"kubernetes.io/projected/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-kube-api-access-8vb2x\") pod \"cluster-monitoring-operator-884576c57-btjpg\" (UID: \"63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg"
Nov 04 02:05:52.699577 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.699555    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcfqp\" (UniqueName: \"kubernetes.io/projected/32699b90-bd7d-43f5-bdcc-53de9da6f244-kube-api-access-qcfqp\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:05:52.699628 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.699583    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7f7m\" (UniqueName: \"kubernetes.io/projected/b2156825-c0d8-48a4-a90f-2340e98a13e3-kube-api-access-b7f7m\") pod \"kas-connection-checker-685895c7c9-pn5gm\" (UID: \"b2156825-c0d8-48a4-a90f-2340e98a13e3\") " pod="kube-system/kas-connection-checker-685895c7c9-pn5gm"
Nov 04 02:05:52.699628 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.699606    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-default-certificate\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:05:52.699718 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.699645    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjm7q\" (UniqueName: \"kubernetes.io/projected/8ec1a73f-2254-499d-ab62-96c8c223ed8d-kube-api-access-kjm7q\") pod \"volume-data-source-validator-58c9f5f46d-xxtnw\" (UID: \"8ec1a73f-2254-499d-ab62-96c8c223ed8d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xxtnw"
Nov 04 02:05:52.699760 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.699727    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32699b90-bd7d-43f5-bdcc-53de9da6f244-service-ca-bundle\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:05:52.707857 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.707829    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Nov 04 02:05:52.707857 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.707841    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Nov 04 02:05:52.707985 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.707894    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Nov 04 02:05:52.707985 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.707894    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 04 02:05:52.707985 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.707927    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 04 02:05:52.707985 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.707933    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 04 02:05:52.708142 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.707991    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 04 02:05:52.708404 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.708392    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Nov 04 02:05:52.709174 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.709157    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"]
Nov 04 02:05:52.709280 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.709267    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-96g6r"
Nov 04 02:05:52.725324 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.725309    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-pxvln"]
Nov 04 02:05:52.725423 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.725409    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"
Nov 04 02:05:52.745815 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.745796    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"]
Nov 04 02:05:52.745907 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.745894    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-pxvln"
Nov 04 02:05:52.748760 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.748736    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Nov 04 02:05:52.748864 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.748817    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 04 02:05:52.748864 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.748826    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 04 02:05:52.749018 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.749002    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-nwfcm\""
Nov 04 02:05:52.749072 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.749007    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Nov 04 02:05:52.749072 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.749011    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Nov 04 02:05:52.749072 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.749042    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-dvtjz\""
Nov 04 02:05:52.749220 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.749080    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wbgvz\""
Nov 04 02:05:52.749220 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.749137    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-n8gzm\""
Nov 04 02:05:52.749220 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.749187    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 04 02:05:52.749318 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.749221    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Nov 04 02:05:52.760156 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.760139    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"]
Nov 04 02:05:52.760268 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.760253    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"
Nov 04 02:05:52.762540 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.762524    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Nov 04 02:05:52.762704 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.762689    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-jnl5r\""
Nov 04 02:05:52.763011 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.762993    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6l5g\" (UniqueName: \"kubernetes.io/projected/abdfaf94-b721-4ed4-8671-416e694df5bb-kube-api-access-l6l5g\") pod \"kas-connection-checker-685895c7c9-2mklg\" (UID: \"abdfaf94-b721-4ed4-8671-416e694df5bb\") " pod="kube-system/kas-connection-checker-685895c7c9-2mklg"
Nov 04 02:05:52.778577 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.778561    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Nov 04 02:05:52.778657 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.778647    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-tdnwm\""
Nov 04 02:05:52.785855 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.785842    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-62blg"]
Nov 04 02:05:52.785972 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.785958    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:05:52.799065 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.799049    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Nov 04 02:05:52.799144 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.799087    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-4gr4n\""
Nov 04 02:05:52.800219 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.800202    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98v7l\" (UniqueName: \"kubernetes.io/projected/3f0365d3-076c-4128-9349-b7dd71181e5f-kube-api-access-98v7l\") pod \"cluster-samples-operator-6ddb6c8b94-h72td\" (UID: \"3f0365d3-076c-4128-9349-b7dd71181e5f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"
Nov 04 02:05:52.800269 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.800239    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cdf1fc0-00ea-4108-8ff0-5c031a193968-service-ca-bundle\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:05:52.800269 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.800257    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-certificates\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:05:52.800332 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.800302    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7cdf1fc0-00ea-4108-8ff0-5c031a193968-tmp\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:05:52.800361 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.800348    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-nginx-conf\") pod \"networking-console-plugin-86c478bb88-pxvln\" (UID: \"0851f8eb-225e-4576-b5bc-dc6d72eefb6c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pxvln"
Nov 04 02:05:52.800392 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.800371    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7f7m\" (UniqueName: \"kubernetes.io/projected/b2156825-c0d8-48a4-a90f-2340e98a13e3-kube-api-access-b7f7m\") pod \"kas-connection-checker-685895c7c9-pn5gm\" (UID: \"b2156825-c0d8-48a4-a90f-2340e98a13e3\") " pod="kube-system/kas-connection-checker-685895c7c9-pn5gm"
Nov 04 02:05:52.800420 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.800389    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-default-certificate\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:05:52.800420 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.800405    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32699b90-bd7d-43f5-bdcc-53de9da6f244-service-ca-bundle\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:05:52.800480 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.800426    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7cdf1fc0-00ea-4108-8ff0-5c031a193968-serving-cert\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:05:52.800480 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.800452    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pxvln\" (UID: \"0851f8eb-225e-4576-b5bc-dc6d72eefb6c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pxvln"
Nov 04 02:05:52.800480 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.800470    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7cdf1fc0-00ea-4108-8ff0-5c031a193968-snapshots\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:05:52.800570 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.800492    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-btjpg\" (UID: \"63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg"
Nov 04 02:05:52.800570 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:52.800530    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/32699b90-bd7d-43f5-bdcc-53de9da6f244-service-ca-bundle podName:32699b90-bd7d-43f5-bdcc-53de9da6f244 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:53.300512245 +0000 UTC m=+32.778599999 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/32699b90-bd7d-43f5-bdcc-53de9da6f244-service-ca-bundle") pod "router-default-6595f8bf85-dmj26" (UID: "32699b90-bd7d-43f5-bdcc-53de9da6f244") : configmap references non-existent config key: service-ca.crt
Nov 04 02:05:52.800674 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.800583    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:05:52.800674 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.800614    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-bound-sa-token\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:05:52.800674 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.800652    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x24q6\" (UniqueName: \"kubernetes.io/projected/a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d-kube-api-access-x24q6\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-96g6r\" (UID: \"a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-96g6r"
Nov 04 02:05:52.800857 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:52.800698    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 02:05:52.800857 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.800702    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vb2x\" (UniqueName: \"kubernetes.io/projected/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-kube-api-access-8vb2x\") pod \"cluster-monitoring-operator-884576c57-btjpg\" (UID: \"63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg"
Nov 04 02:05:52.800857 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:52.800743    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs podName:32699b90-bd7d-43f5-bdcc-53de9da6f244 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:53.300729072 +0000 UTC m=+32.778816838 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs") pod "router-default-6595f8bf85-dmj26" (UID: "32699b90-bd7d-43f5-bdcc-53de9da6f244") : secret "router-metrics-certs-default" not found
Nov 04 02:05:52.800857 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.800770    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcfqp\" (UniqueName: \"kubernetes.io/projected/32699b90-bd7d-43f5-bdcc-53de9da6f244-kube-api-access-qcfqp\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:05:52.800857 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.800797    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:05:52.800857 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.800819    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-h72td\" (UID: \"3f0365d3-076c-4128-9349-b7dd71181e5f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"
Nov 04 02:05:52.800857 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.800844    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cab3ee15-5d68-4840-83f4-ea8e2308e530-image-registry-private-configuration\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:05:52.801232 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.800902    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-96g6r\" (UID: \"a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-96g6r"
Nov 04 02:05:52.801232 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.800977    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjm7q\" (UniqueName: \"kubernetes.io/projected/8ec1a73f-2254-499d-ab62-96c8c223ed8d-kube-api-access-kjm7q\") pod \"volume-data-source-validator-58c9f5f46d-xxtnw\" (UID: \"8ec1a73f-2254-499d-ab62-96c8c223ed8d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xxtnw"
Nov 04 02:05:52.801232 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.801022    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-h72td\" (UID: \"3f0365d3-076c-4128-9349-b7dd71181e5f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"
Nov 04 02:05:52.801232 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.801056    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-96g6r\" (UID: \"a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-96g6r"
Nov 04 02:05:52.801232 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.801084    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-stats-auth\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:05:52.801232 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.801153    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqvtx\" (UniqueName: \"kubernetes.io/projected/1597f8a8-5b13-4076-9d95-c494cf3ad5dc-kube-api-access-nqvtx\") pod \"kas-connection-checker-685895c7c9-d27bw\" (UID: \"1597f8a8-5b13-4076-9d95-c494cf3ad5dc\") " pod="kube-system/kas-connection-checker-685895c7c9-d27bw"
Nov 04 02:05:52.801232 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.801186    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cdf1fc0-00ea-4108-8ff0-5c031a193968-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:05:52.801232 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.801204    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnpf6\" (UniqueName: \"kubernetes.io/projected/7cdf1fc0-00ea-4108-8ff0-5c031a193968-kube-api-access-cnpf6\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:05:52.801232 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.801220    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h77nn\" (UniqueName: \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-kube-api-access-h77nn\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:05:52.801589 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.801240    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cab3ee15-5d68-4840-83f4-ea8e2308e530-installation-pull-secrets\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:05:52.801589 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.801275    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cab3ee15-5d68-4840-83f4-ea8e2308e530-ca-trust-extracted\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:05:52.801589 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.801354    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cab3ee15-5d68-4840-83f4-ea8e2308e530-trusted-ca\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:05:52.801589 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.801369    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-btjpg\" (UID: \"63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg"
Nov 04 02:05:52.801589 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.801384    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-btjpg\" (UID: \"63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg"
Nov 04 02:05:52.801589 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:52.801443    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 02:05:52.801589 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:52.801481    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-cluster-monitoring-operator-tls podName:63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa nodeName:}" failed. No retries permitted until 2026-11-04 02:05:53.301471019 +0000 UTC m=+32.779558774 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-btjpg" (UID: "63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa") : secret "cluster-monitoring-operator-tls" not found
Nov 04 02:05:52.802147 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.802129    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-c2vfk"]
Nov 04 02:05:52.802239 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.802225    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-62blg"
Nov 04 02:05:52.803234 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.803217    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-default-certificate\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:05:52.803323 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.803237    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-stats-auth\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:05:52.805465 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.805440    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Nov 04 02:05:52.805554 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.805464    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 04 02:05:52.805554 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.805481    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Nov 04 02:05:52.805554 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.805486    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Nov 04 02:05:52.805554 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.805510    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Nov 04 02:05:52.805837 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.805601    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 04 02:05:52.805837 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.805619    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Nov 04 02:05:52.805837 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.805631    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Nov 04 02:05:52.805837 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.805726    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 04 02:05:52.805837 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.805759    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Nov 04 02:05:52.824394 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.824374    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gflvk"]
Nov 04 02:05:52.824488 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.824475    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-c2vfk"
Nov 04 02:05:52.838679 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.838657    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8sr5s"]
Nov 04 02:05:52.838765 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.838751    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gflvk"
Nov 04 02:05:52.854810 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.854793    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-2mklg"]
Nov 04 02:05:52.854888 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.854820    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-pn5gm"]
Nov 04 02:05:52.854888 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.854834    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-d27bw"]
Nov 04 02:05:52.854888 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.854847    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xxtnw"]
Nov 04 02:05:52.854888 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.854859    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-pxvln"]
Nov 04 02:05:52.854888 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.854872    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-6cc2t"]
Nov 04 02:05:52.854888 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.854884    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"]
Nov 04 02:05:52.855136 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.854892    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:05:52.855136 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.854900    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6595f8bf85-dmj26"]
Nov 04 02:05:52.855136 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.855029    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"]
Nov 04 02:05:52.855136 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.855062    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-62blg"]
Nov 04 02:05:52.855136 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.855085    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gflvk"]
Nov 04 02:05:52.855136 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.855129    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bd9d74c6b-thtzz"]
Nov 04 02:05:52.855362 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.855148    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg"]
Nov 04 02:05:52.855362 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.855169    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-96g6r"]
Nov 04 02:05:52.855362 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.855190    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"]
Nov 04 02:05:52.855362 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.855207    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8sr5s"]
Nov 04 02:05:52.855362 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.855232    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-c2vfk"]
Nov 04 02:05:52.857083 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.857055    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 04 02:05:52.857243 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.857073    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Nov 04 02:05:52.857243 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.857117    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-4f6g8\""
Nov 04 02:05:52.857243 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.857155    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Nov 04 02:05:52.857421 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.857156    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 04 02:05:52.857421 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.857162    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Nov 04 02:05:52.857421 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.857161    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Nov 04 02:05:52.857915 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.857894    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Nov 04 02:05:52.863055 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.863034    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7f7m\" (UniqueName: \"kubernetes.io/projected/b2156825-c0d8-48a4-a90f-2340e98a13e3-kube-api-access-b7f7m\") pod \"kas-connection-checker-685895c7c9-pn5gm\" (UID: \"b2156825-c0d8-48a4-a90f-2340e98a13e3\") " pod="kube-system/kas-connection-checker-685895c7c9-pn5gm"
Nov 04 02:05:52.863344 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.863326    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqvtx\" (UniqueName: \"kubernetes.io/projected/1597f8a8-5b13-4076-9d95-c494cf3ad5dc-kube-api-access-nqvtx\") pod \"kas-connection-checker-685895c7c9-d27bw\" (UID: \"1597f8a8-5b13-4076-9d95-c494cf3ad5dc\") " pod="kube-system/kas-connection-checker-685895c7c9-d27bw"
Nov 04 02:05:52.863405 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.863369    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vb2x\" (UniqueName: \"kubernetes.io/projected/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-kube-api-access-8vb2x\") pod \"cluster-monitoring-operator-884576c57-btjpg\" (UID: \"63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg"
Nov 04 02:05:52.863440 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.863410    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcfqp\" (UniqueName: \"kubernetes.io/projected/32699b90-bd7d-43f5-bdcc-53de9da6f244-kube-api-access-qcfqp\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:05:52.863475 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.863450    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjm7q\" (UniqueName: \"kubernetes.io/projected/8ec1a73f-2254-499d-ab62-96c8c223ed8d-kube-api-access-kjm7q\") pod \"volume-data-source-validator-58c9f5f46d-xxtnw\" (UID: \"8ec1a73f-2254-499d-ab62-96c8c223ed8d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xxtnw"
Nov 04 02:05:52.876410 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.876366    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-2mklg"
Nov 04 02:05:52.889699 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.889685    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-pn5gm"
Nov 04 02:05:52.902225 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.902201    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-image-registry-private-configuration\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:05:52.902326 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.902238    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7cdf1fc0-00ea-4108-8ff0-5c031a193968-serving-cert\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:05:52.902326 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.902255    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:05:52.902326 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.902312    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pxvln\" (UID: \"0851f8eb-225e-4576-b5bc-dc6d72eefb6c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pxvln"
Nov 04 02:05:52.902489 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.902435    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7cdf1fc0-00ea-4108-8ff0-5c031a193968-snapshots\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:05:52.902489 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:52.902450    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 02:05:52.902489 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.902454    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4gp2\" (UniqueName: \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-kube-api-access-l4gp2\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:05:52.902636 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:52.902497    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert podName:0851f8eb-225e-4576-b5bc-dc6d72eefb6c nodeName:}" failed. No retries permitted until 2026-11-04 02:05:53.402478806 +0000 UTC m=+32.880566572 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-pxvln" (UID: "0851f8eb-225e-4576-b5bc-dc6d72eefb6c") : secret "networking-console-plugin-cert" not found
Nov 04 02:05:52.902636 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.902518    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96kfp\" (UniqueName: \"kubernetes.io/projected/16f00163-6ec3-4c10-81e4-c8a501e15e82-kube-api-access-96kfp\") pod \"console-operator-7cd56b8dbc-jcqmp\" (UID: \"16f00163-6ec3-4c10-81e4-c8a501e15e82\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"
Nov 04 02:05:52.902636 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.902537    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/16f00163-6ec3-4c10-81e4-c8a501e15e82-serving-cert\") pod \"console-operator-7cd56b8dbc-jcqmp\" (UID: \"16f00163-6ec3-4c10-81e4-c8a501e15e82\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"
Nov 04 02:05:52.902753 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.902630    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-bound-sa-token\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:05:52.902753 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.902683    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x24q6\" (UniqueName: \"kubernetes.io/projected/a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d-kube-api-access-x24q6\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-96g6r\" (UID: \"a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-96g6r"
Nov 04 02:05:52.902753 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.902730    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-bound-sa-token\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:05:52.902898 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.902759    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/150189f8-cd39-4a16-9d07-112edc6e83dc-config\") pod \"service-ca-operator-544d749c8c-c2vfk\" (UID: \"150189f8-cd39-4a16-9d07-112edc6e83dc\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-c2vfk"
Nov 04 02:05:52.902898 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.902838    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:05:52.902898 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.902873    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-h72td\" (UID: \"3f0365d3-076c-4128-9349-b7dd71181e5f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"
Nov 04 02:05:52.903035 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.902907    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2v2n\" (UniqueName: \"kubernetes.io/projected/150189f8-cd39-4a16-9d07-112edc6e83dc-kube-api-access-j2v2n\") pod \"service-ca-operator-544d749c8c-c2vfk\" (UID: \"150189f8-cd39-4a16-9d07-112edc6e83dc\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-c2vfk"
Nov 04 02:05:52.903035 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:52.902930    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 02:05:52.903035 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.902935    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cab3ee15-5d68-4840-83f4-ea8e2308e530-image-registry-private-configuration\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:05:52.903035 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.902908    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7cdf1fc0-00ea-4108-8ff0-5c031a193968-snapshots\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:05:52.903035 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:52.902944    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bd9d74c6b-thtzz: secret "image-registry-tls" not found
Nov 04 02:05:52.903035 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.902978    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-96g6r\" (UID: \"a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-96g6r"
Nov 04 02:05:52.903035 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:52.902999    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls podName:cab3ee15-5d68-4840-83f4-ea8e2308e530 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:53.402984492 +0000 UTC m=+32.881072267 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls") pod "image-registry-5bd9d74c6b-thtzz" (UID: "cab3ee15-5d68-4840-83f4-ea8e2308e530") : secret "image-registry-tls" not found
Nov 04 02:05:52.903035 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:52.902950    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 02:05:52.903433 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.903046    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-h72td\" (UID: \"3f0365d3-076c-4128-9349-b7dd71181e5f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"
Nov 04 02:05:52.903433 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:52.903060    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls podName:3f0365d3-076c-4128-9349-b7dd71181e5f nodeName:}" failed. No retries permitted until 2026-11-04 02:05:53.403045036 +0000 UTC m=+32.881132795 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-h72td" (UID: "3f0365d3-076c-4128-9349-b7dd71181e5f") : secret "samples-operator-tls" not found
Nov 04 02:05:52.903433 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.903085    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-installation-pull-secrets\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:05:52.903433 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.903165    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-96g6r\" (UID: \"a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-96g6r"
Nov 04 02:05:52.903433 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.903200    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/150189f8-cd39-4a16-9d07-112edc6e83dc-serving-cert\") pod \"service-ca-operator-544d749c8c-c2vfk\" (UID: \"150189f8-cd39-4a16-9d07-112edc6e83dc\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-c2vfk"
Nov 04 02:05:52.903433 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.903247    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cdf1fc0-00ea-4108-8ff0-5c031a193968-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:05:52.903433 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.903275    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnpf6\" (UniqueName: \"kubernetes.io/projected/7cdf1fc0-00ea-4108-8ff0-5c031a193968-kube-api-access-cnpf6\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:05:52.903433 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.903311    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h77nn\" (UniqueName: \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-kube-api-access-h77nn\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:05:52.903433 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.903359    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-ca-trust-extracted\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:05:52.903433 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.903401    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cab3ee15-5d68-4840-83f4-ea8e2308e530-installation-pull-secrets\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:05:52.903433 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.903433    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cab3ee15-5d68-4840-83f4-ea8e2308e530-ca-trust-extracted\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:05:52.903947 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.903475    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cab3ee15-5d68-4840-83f4-ea8e2308e530-trusted-ca\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:05:52.903947 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.903514    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8st8\" (UniqueName: \"kubernetes.io/projected/301f5fdf-8bd3-49b3-be41-5684c210577c-kube-api-access-k8st8\") pod \"ingress-canary-gflvk\" (UID: \"301f5fdf-8bd3-49b3-be41-5684c210577c\") " pod="openshift-ingress-canary/ingress-canary-gflvk"
Nov 04 02:05:52.903947 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.903540    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/16f00163-6ec3-4c10-81e4-c8a501e15e82-config\") pod \"console-operator-7cd56b8dbc-jcqmp\" (UID: \"16f00163-6ec3-4c10-81e4-c8a501e15e82\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"
Nov 04 02:05:52.903947 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.903570    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98v7l\" (UniqueName: \"kubernetes.io/projected/3f0365d3-076c-4128-9349-b7dd71181e5f-kube-api-access-98v7l\") pod \"cluster-samples-operator-6ddb6c8b94-h72td\" (UID: \"3f0365d3-076c-4128-9349-b7dd71181e5f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"
Nov 04 02:05:52.903947 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.903604    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/301f5fdf-8bd3-49b3-be41-5684c210577c-cert\") pod \"ingress-canary-gflvk\" (UID: \"301f5fdf-8bd3-49b3-be41-5684c210577c\") " pod="openshift-ingress-canary/ingress-canary-gflvk"
Nov 04 02:05:52.903947 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.903645    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cdf1fc0-00ea-4108-8ff0-5c031a193968-service-ca-bundle\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:05:52.903947 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.903674    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-certificates\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:05:52.903947 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.903699    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-certificates\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:05:52.903947 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.903724    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrwzs\" (UniqueName: \"kubernetes.io/projected/37d0fd53-4873-4d8e-9472-6754d577c9a5-kube-api-access-mrwzs\") pod \"network-check-source-6f78d59f99-62blg\" (UID: \"37d0fd53-4873-4d8e-9472-6754d577c9a5\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-62blg"
Nov 04 02:05:52.903947 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.903735    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-96g6r\" (UID: \"a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-96g6r"
Nov 04 02:05:52.903947 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.903751    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7cdf1fc0-00ea-4108-8ff0-5c031a193968-tmp\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:05:52.903947 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.903776    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16f00163-6ec3-4c10-81e4-c8a501e15e82-trusted-ca\") pod \"console-operator-7cd56b8dbc-jcqmp\" (UID: \"16f00163-6ec3-4c10-81e4-c8a501e15e82\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"
Nov 04 02:05:52.903947 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.903781    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-h72td\" (UID: \"3f0365d3-076c-4128-9349-b7dd71181e5f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"
Nov 04 02:05:52.904580 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.904400    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-trusted-ca\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:05:52.904580 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.904488    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cdf1fc0-00ea-4108-8ff0-5c031a193968-service-ca-bundle\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:05:52.908152 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.904731    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7cdf1fc0-00ea-4108-8ff0-5c031a193968-tmp\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:05:52.908152 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.904793    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-nginx-conf\") pod \"networking-console-plugin-86c478bb88-pxvln\" (UID: \"0851f8eb-225e-4576-b5bc-dc6d72eefb6c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pxvln"
Nov 04 02:05:52.908152 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.904984    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cdf1fc0-00ea-4108-8ff0-5c031a193968-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:05:52.908152 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.904994    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-certificates\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:05:52.908152 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.905322    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cab3ee15-5d68-4840-83f4-ea8e2308e530-ca-trust-extracted\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:05:52.908152 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.905512    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-nginx-conf\") pod \"networking-console-plugin-86c478bb88-pxvln\" (UID: \"0851f8eb-225e-4576-b5bc-dc6d72eefb6c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pxvln"
Nov 04 02:05:52.908152 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.905688    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7cdf1fc0-00ea-4108-8ff0-5c031a193968-serving-cert\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:05:52.908152 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.905846    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-96g6r\" (UID: \"a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-96g6r"
Nov 04 02:05:52.908152 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.905891    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cab3ee15-5d68-4840-83f4-ea8e2308e530-trusted-ca\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:05:52.908152 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.905938    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Nov 04 02:05:52.908152 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.905981    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 02:05:52.908152 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.906124    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Nov 04 02:05:52.908152 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.906184    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 04 02:05:52.908152 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.906232    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 04 02:05:52.908152 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.906235    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Nov 04 02:05:52.908152 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.906252    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 02:05:52.908152 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.906352    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cab3ee15-5d68-4840-83f4-ea8e2308e530-installation-pull-secrets\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:05:52.908152 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.907523    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cab3ee15-5d68-4840-83f4-ea8e2308e530-image-registry-private-configuration\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:05:52.908152 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.907699    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 04 02:05:52.910393 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.910372    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-d27bw"
Nov 04 02:05:52.949286 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.949250    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 04 02:05:52.949480 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.949462    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 04 02:05:52.949645 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.949626    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-mbl7d\""
Nov 04 02:05:52.949721 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.949708    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-zd6qr\""
Nov 04 02:05:52.949782 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.949770    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pzhlz\""
Nov 04 02:05:52.949866 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.949853    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-75dtl\""
Nov 04 02:05:52.949918 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.949887    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Nov 04 02:05:52.970359 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.969435    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xxtnw"
Nov 04 02:05:52.977034 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.975319    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnpf6\" (UniqueName: \"kubernetes.io/projected/7cdf1fc0-00ea-4108-8ff0-5c031a193968-kube-api-access-cnpf6\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:05:52.977034 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.975657    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98v7l\" (UniqueName: \"kubernetes.io/projected/3f0365d3-076c-4128-9349-b7dd71181e5f-kube-api-access-98v7l\") pod \"cluster-samples-operator-6ddb6c8b94-h72td\" (UID: \"3f0365d3-076c-4128-9349-b7dd71181e5f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"
Nov 04 02:05:52.977034 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.976172    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x24q6\" (UniqueName: \"kubernetes.io/projected/a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d-kube-api-access-x24q6\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-96g6r\" (UID: \"a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-96g6r"
Nov 04 02:05:52.977034 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.976424    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h77nn\" (UniqueName: \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-kube-api-access-h77nn\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:05:52.977034 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.976978    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-bound-sa-token\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:05:52.981873 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.981347    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:05:52.981873 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.981516    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:52.992985 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:52.990696    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:05:53.007371 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.005840    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-image-registry-private-configuration\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:05:53.007371 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.005869    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:05:53.007371 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.005898    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63254ae3-0393-4094-9401-e18067d30080-tmp-dir\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:05:53.007371 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.005916    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvghh\" (UniqueName: \"kubernetes.io/projected/63254ae3-0393-4094-9401-e18067d30080-kube-api-access-rvghh\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:05:53.007371 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.005935    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4gp2\" (UniqueName: \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-kube-api-access-l4gp2\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:05:53.007371 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.005952    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96kfp\" (UniqueName: \"kubernetes.io/projected/16f00163-6ec3-4c10-81e4-c8a501e15e82-kube-api-access-96kfp\") pod \"console-operator-7cd56b8dbc-jcqmp\" (UID: \"16f00163-6ec3-4c10-81e4-c8a501e15e82\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"
Nov 04 02:05:53.007371 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.005970    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/16f00163-6ec3-4c10-81e4-c8a501e15e82-serving-cert\") pod \"console-operator-7cd56b8dbc-jcqmp\" (UID: \"16f00163-6ec3-4c10-81e4-c8a501e15e82\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"
Nov 04 02:05:53.007371 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.005997    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-bound-sa-token\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:05:53.007371 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.006012    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/150189f8-cd39-4a16-9d07-112edc6e83dc-config\") pod \"service-ca-operator-544d749c8c-c2vfk\" (UID: \"150189f8-cd39-4a16-9d07-112edc6e83dc\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-c2vfk"
Nov 04 02:05:53.007371 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.006042    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2v2n\" (UniqueName: \"kubernetes.io/projected/150189f8-cd39-4a16-9d07-112edc6e83dc-kube-api-access-j2v2n\") pod \"service-ca-operator-544d749c8c-c2vfk\" (UID: \"150189f8-cd39-4a16-9d07-112edc6e83dc\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-c2vfk"
Nov 04 02:05:53.007371 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.006058    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:05:53.007371 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.006082    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-installation-pull-secrets\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:05:53.007371 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.006117    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/150189f8-cd39-4a16-9d07-112edc6e83dc-serving-cert\") pod \"service-ca-operator-544d749c8c-c2vfk\" (UID: \"150189f8-cd39-4a16-9d07-112edc6e83dc\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-c2vfk"
Nov 04 02:05:53.007371 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.006144    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-ca-trust-extracted\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:05:53.007371 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.006160    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/63254ae3-0393-4094-9401-e18067d30080-config-volume\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:05:53.007371 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.006197    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8st8\" (UniqueName: \"kubernetes.io/projected/301f5fdf-8bd3-49b3-be41-5684c210577c-kube-api-access-k8st8\") pod \"ingress-canary-gflvk\" (UID: \"301f5fdf-8bd3-49b3-be41-5684c210577c\") " pod="openshift-ingress-canary/ingress-canary-gflvk"
Nov 04 02:05:53.008148 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.006212    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/16f00163-6ec3-4c10-81e4-c8a501e15e82-config\") pod \"console-operator-7cd56b8dbc-jcqmp\" (UID: \"16f00163-6ec3-4c10-81e4-c8a501e15e82\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"
Nov 04 02:05:53.008148 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.006232    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/301f5fdf-8bd3-49b3-be41-5684c210577c-cert\") pod \"ingress-canary-gflvk\" (UID: \"301f5fdf-8bd3-49b3-be41-5684c210577c\") " pod="openshift-ingress-canary/ingress-canary-gflvk"
Nov 04 02:05:53.008148 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.006259    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-certificates\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:05:53.008148 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.006275    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrwzs\" (UniqueName: \"kubernetes.io/projected/37d0fd53-4873-4d8e-9472-6754d577c9a5-kube-api-access-mrwzs\") pod \"network-check-source-6f78d59f99-62blg\" (UID: \"37d0fd53-4873-4d8e-9472-6754d577c9a5\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-62blg"
Nov 04 02:05:53.008148 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.006290    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16f00163-6ec3-4c10-81e4-c8a501e15e82-trusted-ca\") pod \"console-operator-7cd56b8dbc-jcqmp\" (UID: \"16f00163-6ec3-4c10-81e4-c8a501e15e82\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"
Nov 04 02:05:53.008148 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.006306    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-trusted-ca\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:05:53.008148 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.007038    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-trusted-ca\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:05:53.008148 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:53.007558    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 02:05:53.008148 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:53.007573    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bc84fd8c9-wbbgn: secret "image-registry-tls" not found
Nov 04 02:05:53.008148 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:53.007635    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls podName:12e9dc19-8f06-4b5d-8237-3ed5a449e09f nodeName:}" failed. No retries permitted until 2026-11-04 02:05:53.507615502 +0000 UTC m=+32.985703272 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls") pod "image-registry-6bc84fd8c9-wbbgn" (UID: "12e9dc19-8f06-4b5d-8237-3ed5a449e09f") : secret "image-registry-tls" not found
Nov 04 02:05:53.008606 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.008480    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/16f00163-6ec3-4c10-81e4-c8a501e15e82-config\") pod \"console-operator-7cd56b8dbc-jcqmp\" (UID: \"16f00163-6ec3-4c10-81e4-c8a501e15e82\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"
Nov 04 02:05:53.008660 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.008621    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-certificates\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:05:53.008904 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.008885    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-ca-trust-extracted\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:05:53.009258 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.009236    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/150189f8-cd39-4a16-9d07-112edc6e83dc-config\") pod \"service-ca-operator-544d749c8c-c2vfk\" (UID: \"150189f8-cd39-4a16-9d07-112edc6e83dc\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-c2vfk"
Nov 04 02:05:53.009848 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.009824    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/150189f8-cd39-4a16-9d07-112edc6e83dc-serving-cert\") pod \"service-ca-operator-544d749c8c-c2vfk\" (UID: \"150189f8-cd39-4a16-9d07-112edc6e83dc\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-c2vfk"
Nov 04 02:05:53.009929 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.009848    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16f00163-6ec3-4c10-81e4-c8a501e15e82-trusted-ca\") pod \"console-operator-7cd56b8dbc-jcqmp\" (UID: \"16f00163-6ec3-4c10-81e4-c8a501e15e82\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"
Nov 04 02:05:53.011082 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.011055    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-image-registry-private-configuration\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:05:53.012001 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.011914    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/301f5fdf-8bd3-49b3-be41-5684c210577c-cert\") pod \"ingress-canary-gflvk\" (UID: \"301f5fdf-8bd3-49b3-be41-5684c210577c\") " pod="openshift-ingress-canary/ingress-canary-gflvk"
Nov 04 02:05:53.012001 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.011955    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-installation-pull-secrets\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:05:53.012355 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.012207    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/16f00163-6ec3-4c10-81e4-c8a501e15e82-serving-cert\") pod \"console-operator-7cd56b8dbc-jcqmp\" (UID: \"16f00163-6ec3-4c10-81e4-c8a501e15e82\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"
Nov 04 02:05:53.030771 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.030582    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-96g6r"
Nov 04 02:05:53.050490 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.050457    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-ghp2x\""
Nov 04 02:05:53.050623 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.050607    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-8fdm8\""
Nov 04 02:05:53.050734 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.050721    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 02:05:53.069806 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.069776    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8st8\" (UniqueName: \"kubernetes.io/projected/301f5fdf-8bd3-49b3-be41-5684c210577c-kube-api-access-k8st8\") pod \"ingress-canary-gflvk\" (UID: \"301f5fdf-8bd3-49b3-be41-5684c210577c\") " pod="openshift-ingress-canary/ingress-canary-gflvk"
Nov 04 02:05:53.071397 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.071322    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-bound-sa-token\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:05:53.071859 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.071837    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96kfp\" (UniqueName: \"kubernetes.io/projected/16f00163-6ec3-4c10-81e4-c8a501e15e82-kube-api-access-96kfp\") pod \"console-operator-7cd56b8dbc-jcqmp\" (UID: \"16f00163-6ec3-4c10-81e4-c8a501e15e82\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"
Nov 04 02:05:53.073224 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.072259    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2v2n\" (UniqueName: \"kubernetes.io/projected/150189f8-cd39-4a16-9d07-112edc6e83dc-kube-api-access-j2v2n\") pod \"service-ca-operator-544d749c8c-c2vfk\" (UID: \"150189f8-cd39-4a16-9d07-112edc6e83dc\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-c2vfk"
Nov 04 02:05:53.073224 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.072817    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4gp2\" (UniqueName: \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-kube-api-access-l4gp2\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:05:53.073756 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.073445    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrwzs\" (UniqueName: \"kubernetes.io/projected/37d0fd53-4873-4d8e-9472-6754d577c9a5-kube-api-access-mrwzs\") pod \"network-check-source-6f78d59f99-62blg\" (UID: \"37d0fd53-4873-4d8e-9472-6754d577c9a5\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-62blg"
Nov 04 02:05:53.107612 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.107562    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63254ae3-0393-4094-9401-e18067d30080-tmp-dir\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:05:53.107711 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.107618    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvghh\" (UniqueName: \"kubernetes.io/projected/63254ae3-0393-4094-9401-e18067d30080-kube-api-access-rvghh\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:05:53.107711 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.107699    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:05:53.107812 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.107754    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/63254ae3-0393-4094-9401-e18067d30080-config-volume\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:05:53.108755 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.107962    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63254ae3-0393-4094-9401-e18067d30080-tmp-dir\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:05:53.108755 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:53.107998    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 02:05:53.108755 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:53.108066    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls podName:63254ae3-0393-4094-9401-e18067d30080 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:53.608048846 +0000 UTC m=+33.086136599 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls") pod "dns-default-8sr5s" (UID: "63254ae3-0393-4094-9401-e18067d30080") : secret "dns-default-metrics-tls" not found
Nov 04 02:05:53.108755 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.108673    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/63254ae3-0393-4094-9401-e18067d30080-config-volume\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:05:53.119398 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.119380    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"
Nov 04 02:05:53.131531 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.131482    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-62blg"
Nov 04 02:05:53.160928 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.160888    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-c2vfk"
Nov 04 02:05:53.170313 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.170254    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvghh\" (UniqueName: \"kubernetes.io/projected/63254ae3-0393-4094-9401-e18067d30080-kube-api-access-rvghh\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:05:53.173154 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.173132    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gflvk"
Nov 04 02:05:53.284953 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.284871    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-pn5gm"]
Nov 04 02:05:53.290234 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.290210    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-2mklg"]
Nov 04 02:05:53.309117 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.308114    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-d27bw"]
Nov 04 02:05:53.310239 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.309797    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32699b90-bd7d-43f5-bdcc-53de9da6f244-service-ca-bundle\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:05:53.310239 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.309891    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:05:53.310239 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.310029    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-btjpg\" (UID: \"63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg"
Nov 04 02:05:53.310239 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:53.310229    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 02:05:53.311220 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:53.310265    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 02:05:53.311269 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:53.310309    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/32699b90-bd7d-43f5-bdcc-53de9da6f244-service-ca-bundle podName:32699b90-bd7d-43f5-bdcc-53de9da6f244 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:54.310289207 +0000 UTC m=+33.788376967 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/32699b90-bd7d-43f5-bdcc-53de9da6f244-service-ca-bundle") pod "router-default-6595f8bf85-dmj26" (UID: "32699b90-bd7d-43f5-bdcc-53de9da6f244") : configmap references non-existent config key: service-ca.crt
Nov 04 02:05:53.311313 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:53.311272    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-cluster-monitoring-operator-tls podName:63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa nodeName:}" failed. No retries permitted until 2026-11-04 02:05:54.311253697 +0000 UTC m=+33.789341463 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-btjpg" (UID: "63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa") : secret "cluster-monitoring-operator-tls" not found
Nov 04 02:05:53.311313 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:53.311292    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs podName:32699b90-bd7d-43f5-bdcc-53de9da6f244 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:54.311279979 +0000 UTC m=+33.789367737 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs") pod "router-default-6595f8bf85-dmj26" (UID: "32699b90-bd7d-43f5-bdcc-53de9da6f244") : secret "router-metrics-certs-default" not found
Nov 04 02:05:53.357503 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.357473    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xxtnw"]
Nov 04 02:05:53.368077 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:53.368043    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ec1a73f_2254_499d_ab62_96c8c223ed8d.slice/crio-5daf3f8bae5fdd2aebb2495080ddbb7fbaa802fb0379fbc89f84bb1191c5cae7 WatchSource:0}: Error finding container 5daf3f8bae5fdd2aebb2495080ddbb7fbaa802fb0379fbc89f84bb1191c5cae7: Status 404 returned error can't find the container with id 5daf3f8bae5fdd2aebb2495080ddbb7fbaa802fb0379fbc89f84bb1191c5cae7
Nov 04 02:05:53.392635 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.392562    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-6cc2t"]
Nov 04 02:05:53.411363 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.411325    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pxvln\" (UID: \"0851f8eb-225e-4576-b5bc-dc6d72eefb6c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pxvln"
Nov 04 02:05:53.411461 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.411411    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:05:53.411461 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.411440    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-h72td\" (UID: \"3f0365d3-076c-4128-9349-b7dd71181e5f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"
Nov 04 02:05:53.411674 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:53.411652    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 02:05:53.411744 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:53.411716    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls podName:3f0365d3-076c-4128-9349-b7dd71181e5f nodeName:}" failed. No retries permitted until 2026-11-04 02:05:54.411697444 +0000 UTC m=+33.889785197 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-h72td" (UID: "3f0365d3-076c-4128-9349-b7dd71181e5f") : secret "samples-operator-tls" not found
Nov 04 02:05:53.411804 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:53.411788    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 02:05:53.411856 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:53.411824    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert podName:0851f8eb-225e-4576-b5bc-dc6d72eefb6c nodeName:}" failed. No retries permitted until 2026-11-04 02:05:54.411812899 +0000 UTC m=+33.889900651 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-pxvln" (UID: "0851f8eb-225e-4576-b5bc-dc6d72eefb6c") : secret "networking-console-plugin-cert" not found
Nov 04 02:05:53.411911 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:53.411894    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 02:05:53.411959 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:53.411912    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bd9d74c6b-thtzz: secret "image-registry-tls" not found
Nov 04 02:05:53.412001 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:53.411962    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls podName:cab3ee15-5d68-4840-83f4-ea8e2308e530 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:54.411945925 +0000 UTC m=+33.890033691 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls") pod "image-registry-5bd9d74c6b-thtzz" (UID: "cab3ee15-5d68-4840-83f4-ea8e2308e530") : secret "image-registry-tls" not found
Nov 04 02:05:53.419876 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.419858    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-96g6r"]
Nov 04 02:05:53.422265 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:53.422247    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9b6f7fe_dea9_4dd0_9a5f_7dbc9c68b54d.slice/crio-d2d7df19176a6115af6bea2c4c097c8ae4bbd930b5fc1d1bbab812c714ed3d9e WatchSource:0}: Error finding container d2d7df19176a6115af6bea2c4c097c8ae4bbd930b5fc1d1bbab812c714ed3d9e: Status 404 returned error can't find the container with id d2d7df19176a6115af6bea2c4c097c8ae4bbd930b5fc1d1bbab812c714ed3d9e
Nov 04 02:05:53.502896 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.502868    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"]
Nov 04 02:05:53.504848 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:53.504820    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16f00163_6ec3_4c10_81e4_c8a501e15e82.slice/crio-05121522ad496cc0e0f17bc32485494d97f6ca0c3f3d32d000db77ad5b733c9d WatchSource:0}: Error finding container 05121522ad496cc0e0f17bc32485494d97f6ca0c3f3d32d000db77ad5b733c9d: Status 404 returned error can't find the container with id 05121522ad496cc0e0f17bc32485494d97f6ca0c3f3d32d000db77ad5b733c9d
Nov 04 02:05:53.512423 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.512403    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:05:53.512564 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:53.512547    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 02:05:53.512620 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:53.512567    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bc84fd8c9-wbbgn: secret "image-registry-tls" not found
Nov 04 02:05:53.512668 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:53.512623    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls podName:12e9dc19-8f06-4b5d-8237-3ed5a449e09f nodeName:}" failed. No retries permitted until 2026-11-04 02:05:54.512604645 +0000 UTC m=+33.990692410 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls") pod "image-registry-6bc84fd8c9-wbbgn" (UID: "12e9dc19-8f06-4b5d-8237-3ed5a449e09f") : secret "image-registry-tls" not found
Nov 04 02:05:53.525059 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.525034    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-62blg"]
Nov 04 02:05:53.527264 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:53.527241    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37d0fd53_4873_4d8e_9472_6754d577c9a5.slice/crio-7fcf54c6679939d1561da051a44d5d1f505a57c41aca974fba797ceed6a8a607 WatchSource:0}: Error finding container 7fcf54c6679939d1561da051a44d5d1f505a57c41aca974fba797ceed6a8a607: Status 404 returned error can't find the container with id 7fcf54c6679939d1561da051a44d5d1f505a57c41aca974fba797ceed6a8a607
Nov 04 02:05:53.595886 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.595853    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-c2vfk"]
Nov 04 02:05:53.598562 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:53.598536    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod150189f8_cd39_4a16_9d07_112edc6e83dc.slice/crio-64136c8a4dccf871eac64c818fc3144e56f3395df30849785854fc1b2522e763 WatchSource:0}: Error finding container 64136c8a4dccf871eac64c818fc3144e56f3395df30849785854fc1b2522e763: Status 404 returned error can't find the container with id 64136c8a4dccf871eac64c818fc3144e56f3395df30849785854fc1b2522e763
Nov 04 02:05:53.606975 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.606924    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gflvk"]
Nov 04 02:05:53.608943 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:53.608920    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod301f5fdf_8bd3_49b3_be41_5684c210577c.slice/crio-c8cc6d9aa1515773d997e809800d5d88fc1e54d6fcbf91dbc603822b31dac17d WatchSource:0}: Error finding container c8cc6d9aa1515773d997e809800d5d88fc1e54d6fcbf91dbc603822b31dac17d: Status 404 returned error can't find the container with id c8cc6d9aa1515773d997e809800d5d88fc1e54d6fcbf91dbc603822b31dac17d
Nov 04 02:05:53.613617 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.613600    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:05:53.613714 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:53.613701    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 02:05:53.613756 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:53.613743    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls podName:63254ae3-0393-4094-9401-e18067d30080 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:54.613732102 +0000 UTC m=+34.091819867 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls") pod "dns-default-8sr5s" (UID: "63254ae3-0393-4094-9401-e18067d30080") : secret "dns-default-metrics-tls" not found
Nov 04 02:05:53.981277 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:53.981251    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:05:54.034731 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.034713    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 02:05:54.034942 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.034927    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-h9jgb\""
Nov 04 02:05:54.159451 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.159426    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-62blg" event={"ID":"37d0fd53-4873-4d8e-9472-6754d577c9a5","Type":"ContainerStarted","Data":"7fcf54c6679939d1561da051a44d5d1f505a57c41aca974fba797ceed6a8a607"}
Nov 04 02:05:54.160636 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.160618    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xxtnw" event={"ID":"8ec1a73f-2254-499d-ab62-96c8c223ed8d","Type":"ContainerStarted","Data":"5daf3f8bae5fdd2aebb2495080ddbb7fbaa802fb0379fbc89f84bb1191c5cae7"}
Nov 04 02:05:54.161724 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.161706    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gflvk" event={"ID":"301f5fdf-8bd3-49b3-be41-5684c210577c","Type":"ContainerStarted","Data":"c8cc6d9aa1515773d997e809800d5d88fc1e54d6fcbf91dbc603822b31dac17d"}
Nov 04 02:05:54.162818 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.162800    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp" event={"ID":"16f00163-6ec3-4c10-81e4-c8a501e15e82","Type":"ContainerStarted","Data":"05121522ad496cc0e0f17bc32485494d97f6ca0c3f3d32d000db77ad5b733c9d"}
Nov 04 02:05:54.163862 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.163845    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-6cc2t" event={"ID":"7cdf1fc0-00ea-4108-8ff0-5c031a193968","Type":"ContainerStarted","Data":"145d997b6376851c6386ed644d5bad00dbc508be47e7c780d1294892033289ad"}
Nov 04 02:05:54.165161 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.165142    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-pn5gm" event={"ID":"b2156825-c0d8-48a4-a90f-2340e98a13e3","Type":"ContainerStarted","Data":"1db9870d2a823a41e64e5655345d281d9a7e31b9bb7769f268af6ca38f62c7a6"}
Nov 04 02:05:54.165240 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.165174    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-pn5gm" event={"ID":"b2156825-c0d8-48a4-a90f-2340e98a13e3","Type":"ContainerStarted","Data":"247a62e09494064bc461faadf45e2862c847e8e9c334172d3cef15deee6e0469"}
Nov 04 02:05:54.166172 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.166154    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-c2vfk" event={"ID":"150189f8-cd39-4a16-9d07-112edc6e83dc","Type":"ContainerStarted","Data":"64136c8a4dccf871eac64c818fc3144e56f3395df30849785854fc1b2522e763"}
Nov 04 02:05:54.167193 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.167169    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-96g6r" event={"ID":"a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d","Type":"ContainerStarted","Data":"d2d7df19176a6115af6bea2c4c097c8ae4bbd930b5fc1d1bbab812c714ed3d9e"}
Nov 04 02:05:54.168500 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.168478    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-d27bw" event={"ID":"1597f8a8-5b13-4076-9d95-c494cf3ad5dc","Type":"ContainerStarted","Data":"0a02efa79b01802fa1667d91bc2fdf5bf5e830cf56564a009726c0978b9dda08"}
Nov 04 02:05:54.168593 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.168518    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-d27bw" event={"ID":"1597f8a8-5b13-4076-9d95-c494cf3ad5dc","Type":"ContainerStarted","Data":"bb70b336d8f2a92201bc78704b4f67cf0b024535f11f713cae860d164a359578"}
Nov 04 02:05:54.169990 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.169949    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-2mklg" event={"ID":"abdfaf94-b721-4ed4-8671-416e694df5bb","Type":"ContainerStarted","Data":"4a6e5db26e67baf0bd791244f2557df7e1cfa9c106fcc91d7aac9d427db9bdac"}
Nov 04 02:05:54.170106 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.170021    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-2mklg" event={"ID":"abdfaf94-b721-4ed4-8671-416e694df5bb","Type":"ContainerStarted","Data":"2b730db703d8beaf1d322054237d8680803ef45ed76a4d8ecc2949f5db116de2"}
Nov 04 02:05:54.320378 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.320351    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-btjpg\" (UID: \"63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg"
Nov 04 02:05:54.320519 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.320426    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32699b90-bd7d-43f5-bdcc-53de9da6f244-service-ca-bundle\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:05:54.320587 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:54.320523    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 02:05:54.320587 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.320540    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:05:54.320680 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:54.320589    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/32699b90-bd7d-43f5-bdcc-53de9da6f244-service-ca-bundle podName:32699b90-bd7d-43f5-bdcc-53de9da6f244 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:56.320571433 +0000 UTC m=+35.798659200 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/32699b90-bd7d-43f5-bdcc-53de9da6f244-service-ca-bundle") pod "router-default-6595f8bf85-dmj26" (UID: "32699b90-bd7d-43f5-bdcc-53de9da6f244") : configmap references non-existent config key: service-ca.crt
Nov 04 02:05:54.320680 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:54.320609    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-cluster-monitoring-operator-tls podName:63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa nodeName:}" failed. No retries permitted until 2026-11-04 02:05:56.320599403 +0000 UTC m=+35.798687163 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-btjpg" (UID: "63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa") : secret "cluster-monitoring-operator-tls" not found
Nov 04 02:05:54.320847 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:54.320697    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 02:05:54.320903 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:54.320879    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs podName:32699b90-bd7d-43f5-bdcc-53de9da6f244 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:56.320867386 +0000 UTC m=+35.798955142 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs") pod "router-default-6595f8bf85-dmj26" (UID: "32699b90-bd7d-43f5-bdcc-53de9da6f244") : secret "router-metrics-certs-default" not found
Nov 04 02:05:54.421784 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.421755    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pxvln\" (UID: \"0851f8eb-225e-4576-b5bc-dc6d72eefb6c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pxvln"
Nov 04 02:05:54.422189 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.421850    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:05:54.422189 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.421881    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-h72td\" (UID: \"3f0365d3-076c-4128-9349-b7dd71181e5f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"
Nov 04 02:05:54.422189 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:54.421908    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 02:05:54.422189 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:54.421981    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert podName:0851f8eb-225e-4576-b5bc-dc6d72eefb6c nodeName:}" failed. No retries permitted until 2026-11-04 02:05:56.421963757 +0000 UTC m=+35.900051525 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-pxvln" (UID: "0851f8eb-225e-4576-b5bc-dc6d72eefb6c") : secret "networking-console-plugin-cert" not found
Nov 04 02:05:54.422189 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:54.421996    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 02:05:54.422189 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:54.422004    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 02:05:54.422189 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:54.422022    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bd9d74c6b-thtzz: secret "image-registry-tls" not found
Nov 04 02:05:54.422189 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:54.422044    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls podName:3f0365d3-076c-4128-9349-b7dd71181e5f nodeName:}" failed. No retries permitted until 2026-11-04 02:05:56.422029612 +0000 UTC m=+35.900117381 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-h72td" (UID: "3f0365d3-076c-4128-9349-b7dd71181e5f") : secret "samples-operator-tls" not found
Nov 04 02:05:54.422189 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:54.422068    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls podName:cab3ee15-5d68-4840-83f4-ea8e2308e530 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:56.422054682 +0000 UTC m=+35.900142439 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls") pod "image-registry-5bd9d74c6b-thtzz" (UID: "cab3ee15-5d68-4840-83f4-ea8e2308e530") : secret "image-registry-tls" not found
Nov 04 02:05:54.522543 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.522502    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:05:54.522679 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:54.522645    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 02:05:54.522679 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:54.522660    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bc84fd8c9-wbbgn: secret "image-registry-tls" not found
Nov 04 02:05:54.522768 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:54.522713    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls podName:12e9dc19-8f06-4b5d-8237-3ed5a449e09f nodeName:}" failed. No retries permitted until 2026-11-04 02:05:56.522696678 +0000 UTC m=+36.000784444 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls") pod "image-registry-6bc84fd8c9-wbbgn" (UID: "12e9dc19-8f06-4b5d-8237-3ed5a449e09f") : secret "image-registry-tls" not found
Nov 04 02:05:54.624473 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.624385    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:05:54.624674 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:54.624660    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 02:05:54.624730 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:54.624709    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls podName:63254ae3-0393-4094-9401-e18067d30080 nodeName:}" failed. No retries permitted until 2026-11-04 02:05:56.624693486 +0000 UTC m=+36.102781238 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls") pod "dns-default-8sr5s" (UID: "63254ae3-0393-4094-9401-e18067d30080") : secret "dns-default-metrics-tls" not found
Nov 04 02:05:54.703147 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.703110    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:05:54.733813 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.733762    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-2mklg" podStartSLOduration=427.733745785 podStartE2EDuration="7m7.733745785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:58:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:05:54.732945712 +0000 UTC m=+34.211033485" watchObservedRunningTime="2026-11-04 02:05:54.733745785 +0000 UTC m=+34.211833561"
Nov 04 02:05:54.851794 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.851738    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-pn5gm" podStartSLOduration=427.851718153 podStartE2EDuration="7m7.851718153s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:58:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:05:54.850757605 +0000 UTC m=+34.328845378" watchObservedRunningTime="2026-11-04 02:05:54.851718153 +0000 UTC m=+34.329805928"
Nov 04 02:05:54.928192 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.928111    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs\") pod \"network-metrics-daemon-4v7xm\" (UID: \"afe4b3d8-a18c-4587-9057-83123ffe3347\") " pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:05:54.928192 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.928188    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e164905d-5505-4d20-af2e-b8acd370bf18-original-pull-secret\") pod \"global-pull-secret-syncer-b4wjr\" (UID: \"e164905d-5505-4d20-af2e-b8acd370bf18\") " pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:54.928991 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:54.928968    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 04 02:05:54.929112 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:54.929036    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs podName:afe4b3d8-a18c-4587-9057-83123ffe3347 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:26.929017268 +0000 UTC m=+66.407105026 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs") pod "network-metrics-daemon-4v7xm" (UID: "afe4b3d8-a18c-4587-9057-83123ffe3347") : secret "metrics-daemon-secret" not found
Nov 04 02:05:54.934739 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.934703    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e164905d-5505-4d20-af2e-b8acd370bf18-original-pull-secret\") pod \"global-pull-secret-syncer-b4wjr\" (UID: \"e164905d-5505-4d20-af2e-b8acd370bf18\") " pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:54.969973 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.969927    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-d27bw" podStartSLOduration=427.969913177 podStartE2EDuration="7m7.969913177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:58:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:05:54.969530915 +0000 UTC m=+34.447618690" watchObservedRunningTime="2026-11-04 02:05:54.969913177 +0000 UTC m=+34.448000948"
Nov 04 02:05:54.977190 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:54.977153    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:05:55.029138 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:55.029086    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrkqj\" (UniqueName: \"kubernetes.io/projected/b0a72217-fe1d-4589-9201-651503694cb4-kube-api-access-nrkqj\") pod \"network-check-target-98h6z\" (UID: \"b0a72217-fe1d-4589-9201-651503694cb4\") " pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:05:55.032002 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:55.031969    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrkqj\" (UniqueName: \"kubernetes.io/projected/b0a72217-fe1d-4589-9201-651503694cb4-kube-api-access-nrkqj\") pod \"network-check-target-98h6z\" (UID: \"b0a72217-fe1d-4589-9201-651503694cb4\") " pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:05:55.111614 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:55.111577    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:05:55.128434 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:55.128416    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:05:55.201223 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:55.201186    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:05:55.671120 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:55.670913    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-98h6z"]
Nov 04 02:05:55.673537 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:55.673514    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-b4wjr"]
Nov 04 02:05:55.674301 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:05:55.674272    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0a72217_fe1d_4589_9201_651503694cb4.slice/crio-dc5dff4d3474af86324f5aa9943a05aa786faf6d3c08346b5025e477d181d7af WatchSource:0}: Error finding container dc5dff4d3474af86324f5aa9943a05aa786faf6d3c08346b5025e477d181d7af: Status 404 returned error can't find the container with id dc5dff4d3474af86324f5aa9943a05aa786faf6d3c08346b5025e477d181d7af
Nov 04 02:05:56.178759 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:56.178718    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xxtnw" event={"ID":"8ec1a73f-2254-499d-ab62-96c8c223ed8d","Type":"ContainerStarted","Data":"b340042a91aef1aa0d26ddacb6fb1c2985487bbb646424dfcf135e3c7cb3b73c"}
Nov 04 02:05:56.180118 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:56.180067    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-b4wjr" event={"ID":"e164905d-5505-4d20-af2e-b8acd370bf18","Type":"ContainerStarted","Data":"2910f2ca4280823089ec4bb4ff7dc00bde03f703a0d57668c9b4132ab4e49170"}
Nov 04 02:05:56.181672 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:56.181648    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-98h6z" event={"ID":"b0a72217-fe1d-4589-9201-651503694cb4","Type":"ContainerStarted","Data":"dc5dff4d3474af86324f5aa9943a05aa786faf6d3c08346b5025e477d181d7af"}
Nov 04 02:05:56.341866 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:56.341827    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:05:56.342036 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:56.341984    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-btjpg\" (UID: \"63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg"
Nov 04 02:05:56.342121 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:56.342037    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32699b90-bd7d-43f5-bdcc-53de9da6f244-service-ca-bundle\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:05:56.342207 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:56.342194    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/32699b90-bd7d-43f5-bdcc-53de9da6f244-service-ca-bundle podName:32699b90-bd7d-43f5-bdcc-53de9da6f244 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:00.342180197 +0000 UTC m=+39.820267951 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/32699b90-bd7d-43f5-bdcc-53de9da6f244-service-ca-bundle") pod "router-default-6595f8bf85-dmj26" (UID: "32699b90-bd7d-43f5-bdcc-53de9da6f244") : configmap references non-existent config key: service-ca.crt
Nov 04 02:05:56.342521 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:56.342506    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 02:05:56.342581 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:56.342545    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs podName:32699b90-bd7d-43f5-bdcc-53de9da6f244 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:00.342535405 +0000 UTC m=+39.820623157 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs") pod "router-default-6595f8bf85-dmj26" (UID: "32699b90-bd7d-43f5-bdcc-53de9da6f244") : secret "router-metrics-certs-default" not found
Nov 04 02:05:56.343288 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:56.343243    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 02:05:56.343388 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:56.343293    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-cluster-monitoring-operator-tls podName:63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa nodeName:}" failed. No retries permitted until 2026-11-04 02:06:00.343278934 +0000 UTC m=+39.821366691 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-btjpg" (UID: "63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa") : secret "cluster-monitoring-operator-tls" not found
Nov 04 02:05:56.440187 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:56.440110    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:05:56.442914 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:56.442891    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:05:56.443028 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:56.442920    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-h72td\" (UID: \"3f0365d3-076c-4128-9349-b7dd71181e5f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"
Nov 04 02:05:56.443084 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:56.443036    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pxvln\" (UID: \"0851f8eb-225e-4576-b5bc-dc6d72eefb6c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pxvln"
Nov 04 02:05:56.443084 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:56.443043    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 02:05:56.443084 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:56.443075    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bd9d74c6b-thtzz: secret "image-registry-tls" not found
Nov 04 02:05:56.443233 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:56.443119    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 02:05:56.443233 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:56.443146    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls podName:cab3ee15-5d68-4840-83f4-ea8e2308e530 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:00.44312908 +0000 UTC m=+39.921216836 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls") pod "image-registry-5bd9d74c6b-thtzz" (UID: "cab3ee15-5d68-4840-83f4-ea8e2308e530") : secret "image-registry-tls" not found
Nov 04 02:05:56.443233 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:56.443172    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 02:05:56.443233 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:56.443176    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls podName:3f0365d3-076c-4128-9349-b7dd71181e5f nodeName:}" failed. No retries permitted until 2026-11-04 02:06:00.443159898 +0000 UTC m=+39.921247667 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-h72td" (UID: "3f0365d3-076c-4128-9349-b7dd71181e5f") : secret "samples-operator-tls" not found
Nov 04 02:05:56.443233 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:56.443211    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert podName:0851f8eb-225e-4576-b5bc-dc6d72eefb6c nodeName:}" failed. No retries permitted until 2026-11-04 02:06:00.44320175 +0000 UTC m=+39.921289502 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-pxvln" (UID: "0851f8eb-225e-4576-b5bc-dc6d72eefb6c") : secret "networking-console-plugin-cert" not found
Nov 04 02:05:56.543989 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:56.543954    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:05:56.544141 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:56.544119    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 02:05:56.544141 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:56.544139    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bc84fd8c9-wbbgn: secret "image-registry-tls" not found
Nov 04 02:05:56.544246 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:56.544193    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls podName:12e9dc19-8f06-4b5d-8237-3ed5a449e09f nodeName:}" failed. No retries permitted until 2026-11-04 02:06:00.544177392 +0000 UTC m=+40.022265170 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls") pod "image-registry-6bc84fd8c9-wbbgn" (UID: "12e9dc19-8f06-4b5d-8237-3ed5a449e09f") : secret "image-registry-tls" not found
Nov 04 02:05:56.644481 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:56.644446    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:05:56.644702 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:56.644686    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 02:05:56.644762 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:05:56.644741    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls podName:63254ae3-0393-4094-9401-e18067d30080 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:00.644726667 +0000 UTC m=+40.122814437 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls") pod "dns-default-8sr5s" (UID: "63254ae3-0393-4094-9401-e18067d30080") : secret "dns-default-metrics-tls" not found
Nov 04 02:05:56.662717 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:56.662692    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:05:59.205939 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:59.205243    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-96g6r" event={"ID":"a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d","Type":"ContainerStarted","Data":"006534a5f46ce8db8e724bbecbf66bdf13f290d4edd54f696aa73d49be3d399e"}
Nov 04 02:05:59.324135 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:59.324055    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-96g6r" podStartSLOduration=395.994561354 podStartE2EDuration="6m41.324036618s" totalImagesPullingTime="5.329475264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:05:59.323762071 +0000 UTC m=+38.801849847" watchObservedRunningTime="2026-11-04 02:05:59.324036618 +0000 UTC m=+38.802124393"
Nov 04 02:05:59.325214 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:05:59.325176    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xxtnw" podStartSLOduration=400.39932573 podStartE2EDuration="6m42.325166443s" totalImagesPullingTime="1.925840713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:59:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:05:56.298126192 +0000 UTC m=+35.776213968" watchObservedRunningTime="2026-11-04 02:05:59.325166443 +0000 UTC m=+38.803254221"
Nov 04 02:06:00.212168 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.211824    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-62blg" event={"ID":"37d0fd53-4873-4d8e-9472-6754d577c9a5","Type":"ContainerStarted","Data":"8b763207db91fffc804769373be34e9101c1b83fc74e6dc27cf3258fb6bc864c"}
Nov 04 02:06:00.214594 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.214562    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gflvk" event={"ID":"301f5fdf-8bd3-49b3-be41-5684c210577c","Type":"ContainerStarted","Data":"b14a7d35d3947c3231f508341955d571a6a90cb08258708718f4faca8526b32e"}
Nov 04 02:06:00.217384 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.217361    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-6cc2t" event={"ID":"7cdf1fc0-00ea-4108-8ff0-5c031a193968","Type":"ContainerStarted","Data":"5116f217e0071fb53e23496020a2322a85e2ca08f45dc488b3d0f5c0daa79e7d"}
Nov 04 02:06:00.220415 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.220385    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-98h6z" event={"ID":"b0a72217-fe1d-4589-9201-651503694cb4","Type":"ContainerStarted","Data":"4f33a13bef305f63d5a6216712d92d2179964e82776fa563fad6bfb2ed31b572"}
Nov 04 02:06:00.220512 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.220433    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:06:00.328909 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.328845    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-62blg" podStartSLOduration=394.160196555 podStartE2EDuration="6m40.328827077s" totalImagesPullingTime="6.168630522s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:59:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:00.32791371 +0000 UTC m=+39.806001543" watchObservedRunningTime="2026-11-04 02:06:00.328827077 +0000 UTC m=+39.806914852"
Nov 04 02:06:00.384963 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.384925    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-btjpg\" (UID: \"63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg"
Nov 04 02:06:00.385148 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.385075    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32699b90-bd7d-43f5-bdcc-53de9da6f244-service-ca-bundle\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:06:00.385211 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.385195    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:06:00.385702 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:06:00.385678    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 02:06:00.385772 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:06:00.385688    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/32699b90-bd7d-43f5-bdcc-53de9da6f244-service-ca-bundle podName:32699b90-bd7d-43f5-bdcc-53de9da6f244 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:08.385667753 +0000 UTC m=+47.863755540 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/32699b90-bd7d-43f5-bdcc-53de9da6f244-service-ca-bundle") pod "router-default-6595f8bf85-dmj26" (UID: "32699b90-bd7d-43f5-bdcc-53de9da6f244") : configmap references non-existent config key: service-ca.crt
Nov 04 02:06:00.385772 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:06:00.385760    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-cluster-monitoring-operator-tls podName:63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa nodeName:}" failed. No retries permitted until 2026-11-04 02:06:08.385742686 +0000 UTC m=+47.863830438 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-btjpg" (UID: "63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa") : secret "cluster-monitoring-operator-tls" not found
Nov 04 02:06:00.385881 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:06:00.385797    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 02:06:00.385881 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:06:00.385845    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs podName:32699b90-bd7d-43f5-bdcc-53de9da6f244 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:08.385833329 +0000 UTC m=+47.863921096 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs") pod "router-default-6595f8bf85-dmj26" (UID: "32699b90-bd7d-43f5-bdcc-53de9da6f244") : secret "router-metrics-certs-default" not found
Nov 04 02:06:00.448106 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.447383    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gflvk" podStartSLOduration=2.361319271 podStartE2EDuration="8.447365004s" totalImagesPullingTime="6.086045733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:05:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:00.446146366 +0000 UTC m=+39.924234140" watchObservedRunningTime="2026-11-04 02:06:00.447365004 +0000 UTC m=+39.925452780"
Nov 04 02:06:00.486172 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.485781    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pxvln\" (UID: \"0851f8eb-225e-4576-b5bc-dc6d72eefb6c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pxvln"
Nov 04 02:06:00.486172 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.485854    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:06:00.486172 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.485874    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-h72td\" (UID: \"3f0365d3-076c-4128-9349-b7dd71181e5f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"
Nov 04 02:06:00.486172 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:06:00.486012    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 02:06:00.486172 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:06:00.486071    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls podName:3f0365d3-076c-4128-9349-b7dd71181e5f nodeName:}" failed. No retries permitted until 2026-11-04 02:06:08.486056974 +0000 UTC m=+47.964144728 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-h72td" (UID: "3f0365d3-076c-4128-9349-b7dd71181e5f") : secret "samples-operator-tls" not found
Nov 04 02:06:00.486711 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:06:00.486630    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 02:06:00.486711 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:06:00.486688    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert podName:0851f8eb-225e-4576-b5bc-dc6d72eefb6c nodeName:}" failed. No retries permitted until 2026-11-04 02:06:08.486672808 +0000 UTC m=+47.964760573 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-pxvln" (UID: "0851f8eb-225e-4576-b5bc-dc6d72eefb6c") : secret "networking-console-plugin-cert" not found
Nov 04 02:06:00.486825 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:06:00.486710    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 02:06:00.486825 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:06:00.486730    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bd9d74c6b-thtzz: secret "image-registry-tls" not found
Nov 04 02:06:00.486825 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:06:00.486777    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls podName:cab3ee15-5d68-4840-83f4-ea8e2308e530 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:08.48676007 +0000 UTC m=+47.964847846 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls") pod "image-registry-5bd9d74c6b-thtzz" (UID: "cab3ee15-5d68-4840-83f4-ea8e2308e530") : secret "image-registry-tls" not found
Nov 04 02:06:00.586677 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.586645    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:06:00.586877 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:06:00.586860    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 02:06:00.586943 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:06:00.586880    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bc84fd8c9-wbbgn: secret "image-registry-tls" not found
Nov 04 02:06:00.586943 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:06:00.586935    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls podName:12e9dc19-8f06-4b5d-8237-3ed5a449e09f nodeName:}" failed. No retries permitted until 2026-11-04 02:06:08.586916851 +0000 UTC m=+48.065004611 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls") pod "image-registry-6bc84fd8c9-wbbgn" (UID: "12e9dc19-8f06-4b5d-8237-3ed5a449e09f") : secret "image-registry-tls" not found
Nov 04 02:06:00.600192 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.600158    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:00.687900 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.687851    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:06:00.688070 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:06:00.688005    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 02:06:00.688070 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:06:00.688063    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls podName:63254ae3-0393-4094-9401-e18067d30080 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:08.688049907 +0000 UTC m=+48.166137659 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls") pod "dns-default-8sr5s" (UID: "63254ae3-0393-4094-9401-e18067d30080") : secret "dns-default-metrics-tls" not found
Nov 04 02:06:00.688604 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.688565    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-6cc2t" podStartSLOduration=397.379253503 podStartE2EDuration="6m43.688551529s" totalImagesPullingTime="6.309298026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:59:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:00.569738919 +0000 UTC m=+40.047826694" watchObservedRunningTime="2026-11-04 02:06:00.688551529 +0000 UTC m=+40.166639351"
Nov 04 02:06:00.689455 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.689421    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-98h6z" podStartSLOduration=34.520559729 podStartE2EDuration="38.689414399s" totalImagesPullingTime="4.16885467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:00.688234257 +0000 UTC m=+40.166322032" watchObservedRunningTime="2026-11-04 02:06:00.689414399 +0000 UTC m=+40.167502170"
Nov 04 02:06:00.752249 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.752194    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-dcf2j"]
Nov 04 02:06:00.778015 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.777995    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-nvl6s"]
Nov 04 02:06:00.778173 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.778145    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-dcf2j"
Nov 04 02:06:00.794676 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.794646    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:00.798675 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.798653    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nvl6s"
Nov 04 02:06:00.813230 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.813179    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-dcf2j"]
Nov 04 02:06:00.822079 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.822055    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-nvl6s"]
Nov 04 02:06:00.831874 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.831851    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 04 02:06:00.831986 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.831855    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-mgqv4\""
Nov 04 02:06:00.831986 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.831938    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 04 02:06:00.890243 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.890216    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vg7p\" (UniqueName: \"kubernetes.io/projected/430bb96c-bf05-4a46-bdd3-612bd19a26d8-kube-api-access-6vg7p\") pod \"migrator-66df5d496b-nvl6s\" (UID: \"430bb96c-bf05-4a46-bdd3-612bd19a26d8\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nvl6s"
Nov 04 02:06:00.890365 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.890272    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9j8z\" (UniqueName: \"kubernetes.io/projected/7c365e54-8c98-4271-a66e-9b13fe91afa7-kube-api-access-s9j8z\") pod \"migrator-66df5d496b-dcf2j\" (UID: \"7c365e54-8c98-4271-a66e-9b13fe91afa7\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-dcf2j"
Nov 04 02:06:00.977841 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.977717    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:00.991655 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.991049    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vg7p\" (UniqueName: \"kubernetes.io/projected/430bb96c-bf05-4a46-bdd3-612bd19a26d8-kube-api-access-6vg7p\") pod \"migrator-66df5d496b-nvl6s\" (UID: \"430bb96c-bf05-4a46-bdd3-612bd19a26d8\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nvl6s"
Nov 04 02:06:00.991655 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:00.991128    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9j8z\" (UniqueName: \"kubernetes.io/projected/7c365e54-8c98-4271-a66e-9b13fe91afa7-kube-api-access-s9j8z\") pod \"migrator-66df5d496b-dcf2j\" (UID: \"7c365e54-8c98-4271-a66e-9b13fe91afa7\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-dcf2j"
Nov 04 02:06:01.053300 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.053271    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9j8z\" (UniqueName: \"kubernetes.io/projected/7c365e54-8c98-4271-a66e-9b13fe91afa7-kube-api-access-s9j8z\") pod \"migrator-66df5d496b-dcf2j\" (UID: \"7c365e54-8c98-4271-a66e-9b13fe91afa7\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-dcf2j"
Nov 04 02:06:01.053473 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.053451    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vg7p\" (UniqueName: \"kubernetes.io/projected/430bb96c-bf05-4a46-bdd3-612bd19a26d8-kube-api-access-6vg7p\") pod \"migrator-66df5d496b-nvl6s\" (UID: \"430bb96c-bf05-4a46-bdd3-612bd19a26d8\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nvl6s"
Nov 04 02:06:01.111243 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.111219    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-dcf2j"
Nov 04 02:06:01.127171 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.127155    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nvl6s"
Nov 04 02:06:01.161574 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.161536    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:01.312977 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.312916    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hh6m5"]
Nov 04 02:06:01.329682 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.329655    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:01.382484 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.378591    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hh6m5"]
Nov 04 02:06:01.386281 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.383778    2582 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 04 02:06:01.386281 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.383859    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 04 02:06:01.386281 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.383925    2582 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-j7wvp\""
Nov 04 02:06:01.497934 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.497905    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:01.498050 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.497969    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn4d2\" (UniqueName: \"kubernetes.io/projected/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-kube-api-access-gn4d2\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:01.498164 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.498143    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-data-volume\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:01.498209 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.498172    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:01.498209 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.498199    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-crio-socket\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:01.568999 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.568935    2582 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-xwzqv-7rqm2"]
Nov 04 02:06:01.581486 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.581458    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:01.582049 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.582018    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-xwzqv-7rqm2"
Nov 04 02:06:01.599063 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.599033    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-data-volume\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:01.599182 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.599070    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:01.599182 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.599133    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-crio-socket\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:01.599266 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.599219    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:01.599316 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:06:01.599257    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 02:06:01.599368 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:06:01.599323    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-insights-runtime-extractor-tls podName:c41ceadb-e895-4b54-b3ea-88e0b6aa56d9 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:02.099304469 +0000 UTC m=+41.577392244 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hh6m5" (UID: "c41ceadb-e895-4b54-b3ea-88e0b6aa56d9") : secret "insights-runtime-extractor-tls" not found
Nov 04 02:06:01.599368 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.599262    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn4d2\" (UniqueName: \"kubernetes.io/projected/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-kube-api-access-gn4d2\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:01.599368 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.599334    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-crio-socket\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:01.599522 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.599385    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-data-volume\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:01.599522 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:06:01.599420    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 02:06:01.599522 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:06:01.599485    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-kube-rbac-proxy-cm podName:c41ceadb-e895-4b54-b3ea-88e0b6aa56d9 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:02.099466719 +0000 UTC m=+41.577554484 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hh6m5" (UID: "c41ceadb-e895-4b54-b3ea-88e0b6aa56d9") : configmap "kube-rbac-proxy" not found
Nov 04 02:06:01.630131 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.630106    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-xwzqv-7rqm2"]
Nov 04 02:06:01.661532 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.661503    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn4d2\" (UniqueName: \"kubernetes.io/projected/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-kube-api-access-gn4d2\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:01.696973 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.696949    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-nvl6s"]
Nov 04 02:06:01.699105 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:06:01.699063    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod430bb96c_bf05_4a46_bdd3_612bd19a26d8.slice/crio-c0f566aa8fe4d1aa54333085c0b2dda4804f9e81bf730ae858b8411ca2451e5c WatchSource:0}: Error finding container c0f566aa8fe4d1aa54333085c0b2dda4804f9e81bf730ae858b8411ca2451e5c: Status 404 returned error can't find the container with id c0f566aa8fe4d1aa54333085c0b2dda4804f9e81bf730ae858b8411ca2451e5c
Nov 04 02:06:01.700326 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.700305    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb9e13cc-14be-46c7-bea5-2501b0a85486-trusted-ca-bundle\") pod \"periodic-gathering-xwzqv-7rqm2\" (UID: \"eb9e13cc-14be-46c7-bea5-2501b0a85486\") " pod="openshift-insights/periodic-gathering-xwzqv-7rqm2"
Nov 04 02:06:01.700464 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.700441    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/eb9e13cc-14be-46c7-bea5-2501b0a85486-archives-path\") pod \"periodic-gathering-xwzqv-7rqm2\" (UID: \"eb9e13cc-14be-46c7-bea5-2501b0a85486\") " pod="openshift-insights/periodic-gathering-xwzqv-7rqm2"
Nov 04 02:06:01.700583 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.700566    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb9e13cc-14be-46c7-bea5-2501b0a85486-service-ca-bundle\") pod \"periodic-gathering-xwzqv-7rqm2\" (UID: \"eb9e13cc-14be-46c7-bea5-2501b0a85486\") " pod="openshift-insights/periodic-gathering-xwzqv-7rqm2"
Nov 04 02:06:01.700625 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.700616    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm6tq\" (UniqueName: \"kubernetes.io/projected/eb9e13cc-14be-46c7-bea5-2501b0a85486-kube-api-access-qm6tq\") pod \"periodic-gathering-xwzqv-7rqm2\" (UID: \"eb9e13cc-14be-46c7-bea5-2501b0a85486\") " pod="openshift-insights/periodic-gathering-xwzqv-7rqm2"
Nov 04 02:06:01.746460 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.746403    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-dcf2j"]
Nov 04 02:06:01.748627 ip-10-0-136-63 kubenswrapper[2582]: W1104 02:06:01.748596    2582 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c365e54_8c98_4271_a66e_9b13fe91afa7.slice/crio-c878d0aba7f0a2dd52a414833678ea583c46fb5436076ca931e1c257fa3092f3 WatchSource:0}: Error finding container c878d0aba7f0a2dd52a414833678ea583c46fb5436076ca931e1c257fa3092f3: Status 404 returned error can't find the container with id c878d0aba7f0a2dd52a414833678ea583c46fb5436076ca931e1c257fa3092f3
Nov 04 02:06:01.762803 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.762766    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:01.801565 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.801543    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb9e13cc-14be-46c7-bea5-2501b0a85486-service-ca-bundle\") pod \"periodic-gathering-xwzqv-7rqm2\" (UID: \"eb9e13cc-14be-46c7-bea5-2501b0a85486\") " pod="openshift-insights/periodic-gathering-xwzqv-7rqm2"
Nov 04 02:06:01.801684 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.801588    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm6tq\" (UniqueName: \"kubernetes.io/projected/eb9e13cc-14be-46c7-bea5-2501b0a85486-kube-api-access-qm6tq\") pod \"periodic-gathering-xwzqv-7rqm2\" (UID: \"eb9e13cc-14be-46c7-bea5-2501b0a85486\") " pod="openshift-insights/periodic-gathering-xwzqv-7rqm2"
Nov 04 02:06:01.801885 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.801858    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb9e13cc-14be-46c7-bea5-2501b0a85486-trusted-ca-bundle\") pod \"periodic-gathering-xwzqv-7rqm2\" (UID: \"eb9e13cc-14be-46c7-bea5-2501b0a85486\") " pod="openshift-insights/periodic-gathering-xwzqv-7rqm2"
Nov 04 02:06:01.801998 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.801969    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/eb9e13cc-14be-46c7-bea5-2501b0a85486-archives-path\") pod \"periodic-gathering-xwzqv-7rqm2\" (UID: \"eb9e13cc-14be-46c7-bea5-2501b0a85486\") " pod="openshift-insights/periodic-gathering-xwzqv-7rqm2"
Nov 04 02:06:01.802193 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.802170    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb9e13cc-14be-46c7-bea5-2501b0a85486-service-ca-bundle\") pod \"periodic-gathering-xwzqv-7rqm2\" (UID: \"eb9e13cc-14be-46c7-bea5-2501b0a85486\") " pod="openshift-insights/periodic-gathering-xwzqv-7rqm2"
Nov 04 02:06:01.802395 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.802355    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/eb9e13cc-14be-46c7-bea5-2501b0a85486-archives-path\") pod \"periodic-gathering-xwzqv-7rqm2\" (UID: \"eb9e13cc-14be-46c7-bea5-2501b0a85486\") " pod="openshift-insights/periodic-gathering-xwzqv-7rqm2"
Nov 04 02:06:01.802647 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.802628    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb9e13cc-14be-46c7-bea5-2501b0a85486-trusted-ca-bundle\") pod \"periodic-gathering-xwzqv-7rqm2\" (UID: \"eb9e13cc-14be-46c7-bea5-2501b0a85486\") " pod="openshift-insights/periodic-gathering-xwzqv-7rqm2"
Nov 04 02:06:01.863840 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.863780    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm6tq\" (UniqueName: \"kubernetes.io/projected/eb9e13cc-14be-46c7-bea5-2501b0a85486-kube-api-access-qm6tq\") pod \"periodic-gathering-xwzqv-7rqm2\" (UID: \"eb9e13cc-14be-46c7-bea5-2501b0a85486\") " pod="openshift-insights/periodic-gathering-xwzqv-7rqm2"
Nov 04 02:06:01.906712 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:01.906671    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-xwzqv-7rqm2"
Nov 04 02:06:02.104821 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:02.104787    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:02.104999 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:06:02.104918    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 02:06:02.104999 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:02.104936    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:02.104999 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:06:02.104994    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-kube-rbac-proxy-cm podName:c41ceadb-e895-4b54-b3ea-88e0b6aa56d9 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:03.104974775 +0000 UTC m=+42.583062530 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hh6m5" (UID: "c41ceadb-e895-4b54-b3ea-88e0b6aa56d9") : configmap "kube-rbac-proxy" not found
Nov 04 02:06:02.105180 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:06:02.105043    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 02:06:02.105180 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:06:02.105107    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-insights-runtime-extractor-tls podName:c41ceadb-e895-4b54-b3ea-88e0b6aa56d9 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:03.105078056 +0000 UTC m=+42.583165823 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hh6m5" (UID: "c41ceadb-e895-4b54-b3ea-88e0b6aa56d9") : secret "insights-runtime-extractor-tls" not found
Nov 04 02:06:02.228057 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:02.227964    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-c2vfk" event={"ID":"150189f8-cd39-4a16-9d07-112edc6e83dc","Type":"ContainerStarted","Data":"6628305f8b5bf2ad7a390949b562af58cf024a97af31356d0a6cef6c460a41c4"}
Nov 04 02:06:02.229582 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:02.229551    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nvl6s" event={"ID":"430bb96c-bf05-4a46-bdd3-612bd19a26d8","Type":"ContainerStarted","Data":"c0f566aa8fe4d1aa54333085c0b2dda4804f9e81bf730ae858b8411ca2451e5c"}
Nov 04 02:06:02.230721 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:02.230695    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-dcf2j" event={"ID":"7c365e54-8c98-4271-a66e-9b13fe91afa7","Type":"ContainerStarted","Data":"c878d0aba7f0a2dd52a414833678ea583c46fb5436076ca931e1c257fa3092f3"}
Nov 04 02:06:02.232577 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:02.232551    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jcqmp_16f00163-6ec3-4c10-81e4-c8a501e15e82/console-operator/0.log"
Nov 04 02:06:02.232660 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:02.232594    2582 generic.go:356] "Generic (PLEG): container finished" podID="16f00163-6ec3-4c10-81e4-c8a501e15e82" containerID="4948f13da6beeb46f3b2fc3d3242e973e3934abc93d616cc0453ee73c4db56ff" exitCode=255
Nov 04 02:06:02.232660 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:02.232623    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp" event={"ID":"16f00163-6ec3-4c10-81e4-c8a501e15e82","Type":"ContainerDied","Data":"4948f13da6beeb46f3b2fc3d3242e973e3934abc93d616cc0453ee73c4db56ff"}
Nov 04 02:06:02.232871 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:02.232856    2582 scope.go:122] "RemoveContainer" containerID="4948f13da6beeb46f3b2fc3d3242e973e3934abc93d616cc0453ee73c4db56ff"
Nov 04 02:06:02.303275 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:02.303244    2582 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-xwzqv-7rqm2"]
Nov 04 02:06:02.346494 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:02.346437    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-c2vfk" podStartSLOduration=394.614378883 podStartE2EDuration="6m42.346424148s" totalImagesPullingTime="7.732045265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:59:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:02.345691164 +0000 UTC m=+41.823778936" watchObservedRunningTime="2026-11-04 02:06:02.346424148 +0000 UTC m=+41.824511921"
Nov 04 02:06:02.855948 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:02.855879    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:03.087324 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:03.087231    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:03.116514 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:03.116444    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:03.116650 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:03.116537    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:03.116650 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:06:03.116596    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 02:06:03.116745 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:06:03.116663    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-insights-runtime-extractor-tls podName:c41ceadb-e895-4b54-b3ea-88e0b6aa56d9 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:05.116649158 +0000 UTC m=+44.594736912 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hh6m5" (UID: "c41ceadb-e895-4b54-b3ea-88e0b6aa56d9") : secret "insights-runtime-extractor-tls" not found
Nov 04 02:06:03.117175 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:03.117148    2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:03.120321 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:03.120298    2582 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"
Nov 04 02:06:03.120424 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:03.120378    2582 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"
Nov 04 02:06:03.238400 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:03.238363    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-xwzqv-7rqm2" event={"ID":"eb9e13cc-14be-46c7-bea5-2501b0a85486","Type":"ContainerStarted","Data":"594defed3b74095e62acf6c62217a28e2c22a2c6f42dba6f0f16d064830846a9"}
Nov 04 02:06:03.238573 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:03.238425    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-xwzqv-7rqm2" event={"ID":"eb9e13cc-14be-46c7-bea5-2501b0a85486","Type":"ContainerStarted","Data":"c6d2b1ba17da682935416cb2e38cf5c0b7197bfa360dc9c158dd133b3fc93a77"}
Nov 04 02:06:03.240169 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:03.240146    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jcqmp_16f00163-6ec3-4c10-81e4-c8a501e15e82/console-operator/1.log"
Nov 04 02:06:03.240650 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:03.240629    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jcqmp_16f00163-6ec3-4c10-81e4-c8a501e15e82/console-operator/0.log"
Nov 04 02:06:03.240741 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:03.240668    2582 generic.go:356] "Generic (PLEG): container finished" podID="16f00163-6ec3-4c10-81e4-c8a501e15e82" containerID="54ef74383bad59d1840a87bd68172bf3587b9c10deb740b68015579e57aeebae" exitCode=255
Nov 04 02:06:03.240791 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:03.240743    2582 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp" event={"ID":"16f00163-6ec3-4c10-81e4-c8a501e15e82","Type":"ContainerDied","Data":"54ef74383bad59d1840a87bd68172bf3587b9c10deb740b68015579e57aeebae"}
Nov 04 02:06:03.240791 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:03.240781    2582 scope.go:122] "RemoveContainer" containerID="4948f13da6beeb46f3b2fc3d3242e973e3934abc93d616cc0453ee73c4db56ff"
Nov 04 02:06:03.240991 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:03.240968    2582 scope.go:122] "RemoveContainer" containerID="54ef74383bad59d1840a87bd68172bf3587b9c10deb740b68015579e57aeebae"
Nov 04 02:06:03.241198 ip-10-0-136-63 kubenswrapper[2582]: E1104 02:06:03.241172    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-jcqmp_openshift-console-operator(16f00163-6ec3-4c10-81e4-c8a501e15e82)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp" podUID="16f00163-6ec3-4c10-81e4-c8a501e15e82"
Nov 04 02:06:03.272517 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:03.272480    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:03.355990 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:03.355941    2582 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-xwzqv-7rqm2" podStartSLOduration=2.355927957 podStartE2EDuration="2.355927957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:06:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:03.355484929 +0000 UTC m=+42.833572704" watchObservedRunningTime="2026-11-04 02:06:03.355927957 +0000 UTC m=+42.834015758"
Nov 04 02:06:03.452422 ip-10-0-136-63 kubenswrapper[2582]: I1104 02:06:03.452323    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:03.969441 ip-10-0-136-63 systemd[1]: Stopping Kubernetes Kubelet...
Nov 04 02:06:03.988521 ip-10-0-136-63 systemd[1]: kubelet.service: Deactivated successfully.
Nov 04 02:06:03.988724 ip-10-0-136-63 systemd[1]: Stopped Kubernetes Kubelet.
Nov 04 02:06:03.988960 ip-10-0-136-63 systemd[1]: kubelet.service: Consumed 3.005s CPU time.
Nov 04 02:06:04.008587 ip-10-0-136-63 systemd[1]: Starting Kubernetes Kubelet...
Nov 04 02:06:04.103257 ip-10-0-136-63 kubenswrapper[4969]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 02:06:04.103257 ip-10-0-136-63 kubenswrapper[4969]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 04 02:06:04.103257 ip-10-0-136-63 kubenswrapper[4969]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 02:06:04.103257 ip-10-0-136-63 kubenswrapper[4969]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 02:06:04.105311 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105238    4969 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:06:04.105311 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105312    4969 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:06:04.105386 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105315    4969 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:06:04.105386 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105319    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:06:04.105386 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105321    4969 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:06:04.105386 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105324    4969 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:06:04.105386 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105327    4969 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:06:04.105386 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105330    4969 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:06:04.105386 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105332    4969 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:06:04.105386 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105335    4969 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:06:04.105386 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105338    4969 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:06:04.105386 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105340    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:06:04.105386 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105343    4969 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:06:04.105386 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105346    4969 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:06:04.105386 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105348    4969 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:06:04.105386 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105351    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:06:04.105386 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105356    4969 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:06:04.105386 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105360    4969 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:06:04.105386 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105363    4969 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:06:04.105386 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105366    4969 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:06:04.105386 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105368    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:06:04.105864 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105371    4969 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:06:04.105864 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105374    4969 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:06:04.105864 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105376    4969 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:06:04.105864 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105379    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:06:04.105864 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105382    4969 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:06:04.105864 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105390    4969 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:06:04.105864 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105394    4969 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:06:04.105864 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105397    4969 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:06:04.105864 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105399    4969 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:06:04.105864 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105402    4969 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:06:04.105864 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105405    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:06:04.105864 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105407    4969 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:06:04.105864 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105410    4969 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:06:04.105864 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105413    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:06:04.105864 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105415    4969 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:06:04.105864 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105418    4969 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:06:04.105864 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105420    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:06:04.105864 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105423    4969 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:06:04.105864 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105425    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:06:04.105864 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105428    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:06:04.106376 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105431    4969 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:06:04.106376 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105435    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:06:04.106376 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105438    4969 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:06:04.106376 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105440    4969 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:06:04.106376 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105443    4969 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:06:04.106376 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105446    4969 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:06:04.106376 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105448    4969 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:06:04.106376 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105452    4969 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:06:04.106376 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105454    4969 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:06:04.106376 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105457    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:06:04.106376 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105460    4969 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:06:04.106376 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105463    4969 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:06:04.106376 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105466    4969 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:06:04.106376 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105468    4969 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:06:04.106376 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105471    4969 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:06:04.106376 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105473    4969 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:06:04.106376 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105476    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:06:04.106376 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105479    4969 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:06:04.106376 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105481    4969 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:06:04.106376 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105484    4969 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:06:04.106904 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105487    4969 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:06:04.106904 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105489    4969 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:06:04.106904 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105492    4969 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:06:04.106904 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105494    4969 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:06:04.106904 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105497    4969 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:06:04.106904 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105499    4969 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:06:04.106904 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105502    4969 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:06:04.106904 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105505    4969 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:06:04.106904 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105508    4969 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:06:04.106904 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105511    4969 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:06:04.106904 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105513    4969 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:06:04.106904 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105516    4969 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:06:04.106904 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105518    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:06:04.106904 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105521    4969 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:06:04.106904 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105523    4969 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:06:04.106904 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105527    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:06:04.106904 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105529    4969 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:06:04.106904 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105532    4969 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:06:04.106904 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105535    4969 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:06:04.107387 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105538    4969 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:06:04.107387 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105541    4969 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:06:04.107387 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105543    4969 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:06:04.107387 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105546    4969 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:06:04.107387 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105549    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:06:04.107387 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105551    4969 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:06:04.107387 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105554    4969 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:06:04.107387 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105556    4969 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:06:04.107387 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105559    4969 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:06:04.107387 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105561    4969 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:06:04.107387 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105564    4969 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:06:04.107387 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105567    4969 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:06:04.107387 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105569    4969 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:06:04.107387 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105572    4969 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:06:04.107387 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105574    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:06:04.107387 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105577    4969 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:06:04.107387 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105579    4969 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:06:04.107387 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105582    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:06:04.107387 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105584    4969 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:06:04.107387 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105587    4969 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:06:04.107886 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105589    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:06:04.107886 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105593    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:06:04.107886 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105595    4969 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:06:04.107886 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105598    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:06:04.107886 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105601    4969 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:06:04.107886 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.105603    4969 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:06:04.107886 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106369    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:06:04.107886 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106377    4969 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:06:04.107886 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106380    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:06:04.107886 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106389    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:06:04.107886 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106394    4969 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:06:04.107886 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106399    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:06:04.107886 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106403    4969 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:06:04.107886 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106406    4969 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:06:04.107886 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106409    4969 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:06:04.107886 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106412    4969 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:06:04.107886 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106415    4969 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:06:04.107886 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106417    4969 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:06:04.107886 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106419    4969 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:06:04.108390 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106424    4969 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:06:04.108390 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106428    4969 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:06:04.108390 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106431    4969 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:06:04.108390 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106433    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:06:04.108390 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106436    4969 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:06:04.108390 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106439    4969 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:06:04.108390 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106441    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:06:04.108390 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106444    4969 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:06:04.108390 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106447    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:06:04.108390 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106449    4969 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:06:04.108390 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106452    4969 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:06:04.108390 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106455    4969 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:06:04.108390 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106457    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:06:04.108390 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106460    4969 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:06:04.108390 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106463    4969 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:06:04.108390 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106466    4969 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:06:04.108390 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106469    4969 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:06:04.108390 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106472    4969 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:06:04.108390 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106475    4969 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:06:04.108860 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106478    4969 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:06:04.108860 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106483    4969 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:06:04.108860 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106487    4969 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:06:04.108860 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106491    4969 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:06:04.108860 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106502    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:06:04.108860 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106508    4969 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:06:04.108860 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106512    4969 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:06:04.108860 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106516    4969 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:06:04.108860 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106520    4969 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:06:04.108860 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106524    4969 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:06:04.108860 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106528    4969 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:06:04.108860 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106532    4969 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:06:04.108860 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106536    4969 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:06:04.108860 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106540    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:06:04.108860 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106545    4969 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:06:04.108860 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106549    4969 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:06:04.108860 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106553    4969 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:06:04.108860 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106557    4969 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:06:04.108860 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106561    4969 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:06:04.108860 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106566    4969 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:06:04.109367 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106571    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:06:04.109367 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106575    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:06:04.109367 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106579    4969 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:06:04.109367 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106583    4969 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:06:04.109367 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106588    4969 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:06:04.109367 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106593    4969 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:06:04.109367 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106597    4969 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:06:04.109367 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106601    4969 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:06:04.109367 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106604    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:06:04.109367 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106606    4969 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:06:04.109367 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106609    4969 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:06:04.109367 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106612    4969 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:06:04.109367 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106617    4969 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:06:04.109367 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106619    4969 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:06:04.109367 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106622    4969 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:06:04.109367 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106625    4969 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:06:04.109367 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106628    4969 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:06:04.109367 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106637    4969 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:06:04.109367 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106640    4969 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:06:04.109367 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106643    4969 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:06:04.109932 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106645    4969 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:06:04.109932 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106649    4969 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:06:04.109932 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106652    4969 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:06:04.109932 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106654    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:06:04.109932 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106657    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:06:04.109932 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106659    4969 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:06:04.109932 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106662    4969 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:06:04.109932 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106664    4969 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:06:04.109932 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106667    4969 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:06:04.109932 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106669    4969 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:06:04.109932 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106672    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:06:04.109932 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106675    4969 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:06:04.109932 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106678    4969 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:06:04.109932 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106680    4969 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:06:04.109932 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106688    4969 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:06:04.109932 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106691    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:06:04.109932 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106694    4969 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:06:04.109932 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106696    4969 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:06:04.109932 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106699    4969 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:06:04.109932 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106701    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:06:04.110688 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106704    4969 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:06:04.110688 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106706    4969 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:06:04.110688 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106709    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:06:04.110688 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106711    4969 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:06:04.110688 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106716    4969 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:06:04.110688 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106719    4969 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:06:04.110688 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106721    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:06:04.110688 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106724    4969 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:06:04.110688 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106726    4969 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:06:04.110688 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106729    4969 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:06:04.110688 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106732    4969 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:06:04.110688 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106735    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:06:04.110688 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106737    4969 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:06:04.110688 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.106740    4969 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:06:04.110688 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107233    4969 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:06:04.110688 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107242    4969 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:06:04.110688 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107247    4969 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:06:04.110688 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107252    4969 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:06:04.110688 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107256    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:06:04.111212 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107259    4969 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:06:04.111212 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107262    4969 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:06:04.111212 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107265    4969 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:06:04.111212 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107267    4969 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:06:04.111212 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107270    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:06:04.111212 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107273    4969 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:06:04.111212 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107276    4969 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:06:04.111212 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107278    4969 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:06:04.111212 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107281    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:06:04.111212 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107283    4969 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:06:04.111212 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107286    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:06:04.111212 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107289    4969 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:06:04.111212 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107291    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:06:04.111212 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107294    4969 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:06:04.111212 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107297    4969 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:06:04.111212 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107300    4969 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:06:04.111212 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107302    4969 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:06:04.111212 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107307    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:06:04.111212 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107310    4969 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:06:04.111720 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107312    4969 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:06:04.111720 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107315    4969 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:06:04.111720 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107317    4969 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:06:04.111720 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107320    4969 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:06:04.111720 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107323    4969 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:06:04.111720 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107326    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:06:04.111720 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107329    4969 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:06:04.111720 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107331    4969 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:06:04.111720 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107334    4969 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:06:04.111720 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107337    4969 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:06:04.111720 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107340    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:06:04.111720 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107342    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:06:04.111720 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107345    4969 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:06:04.111720 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107347    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:06:04.111720 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107350    4969 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:06:04.111720 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107352    4969 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:06:04.111720 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107355    4969 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:06:04.111720 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107357    4969 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:06:04.111720 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107360    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:06:04.111720 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107363    4969 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:06:04.112231 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107365    4969 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:06:04.112231 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107368    4969 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:06:04.112231 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107370    4969 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:06:04.112231 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107374    4969 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:06:04.112231 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107376    4969 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:06:04.112231 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107379    4969 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:06:04.112231 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107381    4969 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:06:04.112231 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107384    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:06:04.112231 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107387    4969 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:06:04.112231 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107389    4969 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:06:04.112231 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107395    4969 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:06:04.112231 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107398    4969 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:06:04.112231 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107400    4969 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:06:04.112231 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107403    4969 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:06:04.112231 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107406    4969 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:06:04.112231 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107408    4969 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:06:04.112231 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107411    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:06:04.112231 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107414    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:06:04.112231 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107417    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:06:04.112231 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107419    4969 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:06:04.112702 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107422    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:06:04.112702 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107424    4969 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:06:04.112702 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107427    4969 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:06:04.112702 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107430    4969 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:06:04.112702 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107432    4969 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:06:04.112702 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107435    4969 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:06:04.112702 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107437    4969 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:06:04.112702 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107440    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:06:04.112702 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107442    4969 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:06:04.112702 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107445    4969 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:06:04.112702 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107448    4969 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:06:04.112702 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107452    4969 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:06:04.112702 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107456    4969 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:06:04.112702 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107459    4969 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:06:04.112702 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107462    4969 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:06:04.112702 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107465    4969 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:06:04.112702 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107467    4969 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:06:04.112702 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107470    4969 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:06:04.112702 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107473    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:06:04.113210 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107475    4969 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:06:04.113210 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107478    4969 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:06:04.113210 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107480    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:06:04.113210 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107484    4969 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:06:04.113210 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107487    4969 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:06:04.113210 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107490    4969 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:06:04.113210 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107492    4969 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:06:04.113210 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107495    4969 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:06:04.113210 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107498    4969 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:06:04.113210 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107500    4969 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:06:04.113210 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107503    4969 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:06:04.113210 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107506    4969 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:06:04.113210 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107509    4969 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:06:04.113210 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107511    4969 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:06:04.113210 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107514    4969 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:06:04.113210 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107516    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:06:04.113210 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107519    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:06:04.113210 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107522    4969 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:06:04.113210 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107524    4969 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:06:04.113210 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107527    4969 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:06:04.113696 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107530    4969 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:06:04.113696 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107532    4969 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:06:04.113696 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.107535    4969 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:06:04.113696 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107635    4969 flags.go:64] FLAG: --address="0.0.0.0"
Nov 04 02:06:04.113696 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107643    4969 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 04 02:06:04.113696 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107648    4969 flags.go:64] FLAG: --anonymous-auth="true"
Nov 04 02:06:04.113696 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107653    4969 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 04 02:06:04.113696 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107657    4969 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 04 02:06:04.113696 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107661    4969 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 04 02:06:04.113696 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107665    4969 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 04 02:06:04.113696 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107669    4969 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 04 02:06:04.113696 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107672    4969 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 04 02:06:04.113696 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107676    4969 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 04 02:06:04.113696 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107680    4969 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 04 02:06:04.113696 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107683    4969 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 04 02:06:04.113696 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107688    4969 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 04 02:06:04.113696 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107692    4969 flags.go:64] FLAG: --cgroup-root=""
Nov 04 02:06:04.113696 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107695    4969 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 04 02:06:04.113696 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107698    4969 flags.go:64] FLAG: --client-ca-file=""
Nov 04 02:06:04.113696 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107702    4969 flags.go:64] FLAG: --cloud-provider="external"
Nov 04 02:06:04.113696 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107705    4969 flags.go:64] FLAG: --cluster-dns="[]"
Nov 04 02:06:04.113696 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107710    4969 flags.go:64] FLAG: --cluster-domain=""
Nov 04 02:06:04.113696 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107713    4969 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 04 02:06:04.113696 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107716    4969 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 04 02:06:04.114289 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107720    4969 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 04 02:06:04.114289 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107723    4969 flags.go:64] FLAG: --container-log-max-files="5"
Nov 04 02:06:04.114289 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107727    4969 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 04 02:06:04.114289 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107730    4969 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 04 02:06:04.114289 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107735    4969 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 04 02:06:04.114289 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107738    4969 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 04 02:06:04.114289 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107741    4969 flags.go:64] FLAG: --contention-profiling="false"
Nov 04 02:06:04.114289 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107744    4969 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 04 02:06:04.114289 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107747    4969 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 04 02:06:04.114289 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107750    4969 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 04 02:06:04.114289 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107753    4969 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 04 02:06:04.114289 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107758    4969 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 04 02:06:04.114289 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107761    4969 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 04 02:06:04.114289 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107764    4969 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 04 02:06:04.114289 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107767    4969 flags.go:64] FLAG: --enable-load-reader="false"
Nov 04 02:06:04.114289 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107770    4969 flags.go:64] FLAG: --enable-server="true"
Nov 04 02:06:04.114289 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107773    4969 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 04 02:06:04.114289 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107777    4969 flags.go:64] FLAG: --event-burst="100"
Nov 04 02:06:04.114289 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107780    4969 flags.go:64] FLAG: --event-qps="50"
Nov 04 02:06:04.114289 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107784    4969 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 04 02:06:04.114289 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107787    4969 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 04 02:06:04.114289 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107790    4969 flags.go:64] FLAG: --eviction-hard=""
Nov 04 02:06:04.114289 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107794    4969 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 04 02:06:04.114289 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107799    4969 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 04 02:06:04.114858 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107802    4969 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 04 02:06:04.114858 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107805    4969 flags.go:64] FLAG: --eviction-soft=""
Nov 04 02:06:04.114858 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107808    4969 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 04 02:06:04.114858 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107811    4969 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 04 02:06:04.114858 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107814    4969 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 04 02:06:04.114858 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107817    4969 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 04 02:06:04.114858 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107820    4969 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 04 02:06:04.114858 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107823    4969 flags.go:64] FLAG: --fail-swap-on="true"
Nov 04 02:06:04.114858 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107826    4969 flags.go:64] FLAG: --feature-gates=""
Nov 04 02:06:04.114858 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107830    4969 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 04 02:06:04.114858 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107834    4969 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 04 02:06:04.114858 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107837    4969 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 04 02:06:04.114858 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107841    4969 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 04 02:06:04.114858 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107844    4969 flags.go:64] FLAG: --healthz-port="10248"
Nov 04 02:06:04.114858 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107847    4969 flags.go:64] FLAG: --help="false"
Nov 04 02:06:04.114858 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107850    4969 flags.go:64] FLAG: --hostname-override="ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:06:04.114858 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107854    4969 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 04 02:06:04.114858 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107857    4969 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 04 02:06:04.114858 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107860    4969 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 04 02:06:04.114858 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107863    4969 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 04 02:06:04.114858 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107867    4969 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 04 02:06:04.114858 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107870    4969 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 04 02:06:04.114858 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107873    4969 flags.go:64] FLAG: --image-service-endpoint=""
Nov 04 02:06:04.115419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107876    4969 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 04 02:06:04.115419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107879    4969 flags.go:64] FLAG: --kube-api-burst="100"
Nov 04 02:06:04.115419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107882    4969 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 04 02:06:04.115419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107885    4969 flags.go:64] FLAG: --kube-api-qps="50"
Nov 04 02:06:04.115419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107888    4969 flags.go:64] FLAG: --kube-reserved=""
Nov 04 02:06:04.115419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107891    4969 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 04 02:06:04.115419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107895    4969 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 04 02:06:04.115419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107898    4969 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 04 02:06:04.115419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107902    4969 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 04 02:06:04.115419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107905    4969 flags.go:64] FLAG: --lock-file=""
Nov 04 02:06:04.115419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107908    4969 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 04 02:06:04.115419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107911    4969 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 04 02:06:04.115419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107914    4969 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 04 02:06:04.115419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107919    4969 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 04 02:06:04.115419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107922    4969 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 04 02:06:04.115419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107925    4969 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 04 02:06:04.115419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107928    4969 flags.go:64] FLAG: --logging-format="text"
Nov 04 02:06:04.115419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107931    4969 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 04 02:06:04.115419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107934    4969 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 04 02:06:04.115419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107938    4969 flags.go:64] FLAG: --manifest-url=""
Nov 04 02:06:04.115419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107941    4969 flags.go:64] FLAG: --manifest-url-header=""
Nov 04 02:06:04.115419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107947    4969 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 04 02:06:04.115419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107950    4969 flags.go:64] FLAG: --max-open-files="1000000"
Nov 04 02:06:04.115419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107954    4969 flags.go:64] FLAG: --max-pods="110"
Nov 04 02:06:04.115419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107957    4969 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 04 02:06:04.116202 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107960    4969 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 04 02:06:04.116202 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107963    4969 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 04 02:06:04.116202 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107966    4969 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 04 02:06:04.116202 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107969    4969 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 04 02:06:04.116202 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107972    4969 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 04 02:06:04.116202 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107976    4969 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 04 02:06:04.116202 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107982    4969 flags.go:64] FLAG: --node-status-max-images="50"
Nov 04 02:06:04.116202 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107985    4969 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 04 02:06:04.116202 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107988    4969 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 04 02:06:04.116202 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107991    4969 flags.go:64] FLAG: --pod-cidr=""
Nov 04 02:06:04.116202 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107994    4969 flags.go:64] FLAG: --pod-manifest-path=""
Nov 04 02:06:04.116202 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.107997    4969 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 04 02:06:04.116202 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108000    4969 flags.go:64] FLAG: --pods-per-core="0"
Nov 04 02:06:04.116202 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108003    4969 flags.go:64] FLAG: --port="10250"
Nov 04 02:06:04.116202 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108006    4969 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 04 02:06:04.116202 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108010    4969 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-066e77ac155495e22"
Nov 04 02:06:04.116202 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108014    4969 flags.go:64] FLAG: --qos-reserved=""
Nov 04 02:06:04.116202 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108017    4969 flags.go:64] FLAG: --read-only-port="10255"
Nov 04 02:06:04.116202 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108020    4969 flags.go:64] FLAG: --register-node="true"
Nov 04 02:06:04.116202 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108023    4969 flags.go:64] FLAG: --register-with-taints=""
Nov 04 02:06:04.116202 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108026    4969 flags.go:64] FLAG: --registry-burst="10"
Nov 04 02:06:04.116202 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108029    4969 flags.go:64] FLAG: --registry-qps="5"
Nov 04 02:06:04.116202 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108032    4969 flags.go:64] FLAG: --reserved-cpus=""
Nov 04 02:06:04.116202 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108035    4969 flags.go:64] FLAG: --reserved-memory=""
Nov 04 02:06:04.116202 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108039    4969 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 04 02:06:04.116202 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108042    4969 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 04 02:06:04.117278 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108045    4969 flags.go:64] FLAG: --rotate-certificates="false"
Nov 04 02:06:04.117278 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108049    4969 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 04 02:06:04.117278 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108053    4969 flags.go:64] FLAG: --runonce="false"
Nov 04 02:06:04.117278 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108056    4969 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 04 02:06:04.117278 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108059    4969 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 04 02:06:04.117278 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108063    4969 flags.go:64] FLAG: --seccomp-default="false"
Nov 04 02:06:04.117278 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108066    4969 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 04 02:06:04.117278 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108068    4969 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 04 02:06:04.117278 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108072    4969 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 04 02:06:04.117278 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108075    4969 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 04 02:06:04.117278 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108078    4969 flags.go:64] FLAG: --storage-driver-password="root"
Nov 04 02:06:04.117278 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108081    4969 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 04 02:06:04.117278 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108084    4969 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 04 02:06:04.117278 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108087    4969 flags.go:64] FLAG: --storage-driver-user="root"
Nov 04 02:06:04.117278 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108103    4969 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 04 02:06:04.117278 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108106    4969 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 04 02:06:04.117278 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108110    4969 flags.go:64] FLAG: --system-cgroups=""
Nov 04 02:06:04.117278 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108112    4969 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 04 02:06:04.117278 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108118    4969 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 04 02:06:04.117278 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108120    4969 flags.go:64] FLAG: --tls-cert-file=""
Nov 04 02:06:04.117278 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108124    4969 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 04 02:06:04.117278 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108129    4969 flags.go:64] FLAG: --tls-min-version=""
Nov 04 02:06:04.117278 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108132    4969 flags.go:64] FLAG: --tls-private-key-file=""
Nov 04 02:06:04.117278 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108135    4969 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 04 02:06:04.117278 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108138    4969 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 04 02:06:04.118067 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108141    4969 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 04 02:06:04.118067 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108144    4969 flags.go:64] FLAG: --v="2"
Nov 04 02:06:04.118067 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108148    4969 flags.go:64] FLAG: --version="false"
Nov 04 02:06:04.118067 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108153    4969 flags.go:64] FLAG: --vmodule=""
Nov 04 02:06:04.118067 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108157    4969 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 04 02:06:04.118067 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108160    4969 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 04 02:06:04.118067 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108289    4969 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:06:04.118067 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108294    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:06:04.118067 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108297    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:06:04.118067 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108302    4969 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:06:04.118067 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108305    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:06:04.118067 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108308    4969 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:06:04.118067 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108311    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:06:04.118067 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108313    4969 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:06:04.118067 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108316    4969 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:06:04.118067 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108319    4969 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:06:04.118067 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108322    4969 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:06:04.118067 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108324    4969 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:06:04.118067 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108327    4969 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:06:04.118067 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108329    4969 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:06:04.118067 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108332    4969 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:06:04.118067 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108335    4969 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:06:04.118778 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108338    4969 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:06:04.118778 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108340    4969 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:06:04.118778 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108343    4969 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:06:04.118778 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108346    4969 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:06:04.118778 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108348    4969 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:06:04.118778 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108351    4969 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:06:04.118778 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108354    4969 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:06:04.118778 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108357    4969 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:06:04.118778 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108360    4969 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:06:04.118778 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108362    4969 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:06:04.118778 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108365    4969 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:06:04.118778 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108368    4969 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:06:04.118778 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108370    4969 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:06:04.118778 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108373    4969 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:06:04.118778 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108376    4969 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:06:04.118778 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108378    4969 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:06:04.118778 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108381    4969 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:06:04.118778 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108383    4969 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:06:04.118778 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108386    4969 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:06:04.118778 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108390    4969 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:06:04.119431 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108393    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:06:04.119431 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108396    4969 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:06:04.119431 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108399    4969 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:06:04.119431 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108402    4969 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:06:04.119431 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108405    4969 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:06:04.119431 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108407    4969 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:06:04.119431 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108410    4969 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:06:04.119431 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108412    4969 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:06:04.119431 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108415    4969 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:06:04.119431 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108417    4969 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:06:04.119431 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108420    4969 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:06:04.119431 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108423    4969 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:06:04.119431 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108426    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:06:04.119431 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108428    4969 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:06:04.119431 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108431    4969 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:06:04.119431 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108434    4969 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:06:04.119431 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108436    4969 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:06:04.119431 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108439    4969 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:06:04.119431 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108443    4969 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:06:04.119989 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108447    4969 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:06:04.119989 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108451    4969 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:06:04.119989 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108455    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:06:04.119989 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108458    4969 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:06:04.119989 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108461    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:06:04.119989 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108464    4969 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:06:04.119989 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108467    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:06:04.119989 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108470    4969 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:06:04.119989 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108473    4969 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:06:04.119989 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108475    4969 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:06:04.119989 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108478    4969 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:06:04.119989 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108481    4969 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:06:04.119989 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108484    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:06:04.119989 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108487    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:06:04.119989 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108490    4969 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:06:04.119989 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108493    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:06:04.119989 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108495    4969 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:06:04.119989 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108498    4969 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:06:04.119989 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108500    4969 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:06:04.120575 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108503    4969 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:06:04.120575 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108506    4969 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:06:04.120575 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108508    4969 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:06:04.120575 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108511    4969 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:06:04.120575 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108513    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:06:04.120575 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108516    4969 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:06:04.120575 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108519    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:06:04.120575 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108521    4969 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:06:04.120575 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108524    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:06:04.120575 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108527    4969 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:06:04.120575 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108529    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:06:04.120575 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108532    4969 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:06:04.120575 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108535    4969 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:06:04.120575 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108537    4969 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:06:04.120575 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108540    4969 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:06:04.120575 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108542    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:06:04.120575 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108545    4969 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:06:04.120575 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108547    4969 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:06:04.120575 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108550    4969 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:06:04.120575 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108553    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:06:04.121318 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108555    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:06:04.121318 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108558    4969 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:06:04.121318 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108560    4969 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:06:04.121318 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108563    4969 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:06:04.121318 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108565    4969 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:06:04.121318 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108569    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:06:04.121318 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108572    4969 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:06:04.121318 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108574    4969 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:06:04.121318 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108578    4969 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:06:04.121318 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108581    4969 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:06:04.121318 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108583    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:06:04.121318 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108586    4969 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:06:04.121318 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.108611    4969 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 02:06:04.121318 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108658    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:06:04.121318 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108665    4969 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:06:04.121318 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108670    4969 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:06:04.121760 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108675    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:06:04.121760 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108679    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:06:04.121760 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108684    4969 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:06:04.121760 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108688    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:06:04.121760 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108693    4969 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:06:04.121760 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108696    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:06:04.121760 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108699    4969 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:06:04.121760 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108701    4969 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:06:04.121760 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108704    4969 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:06:04.121760 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108707    4969 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:06:04.121760 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108709    4969 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:06:04.121760 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108712    4969 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:06:04.121760 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108715    4969 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:06:04.121760 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108717    4969 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:06:04.121760 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108720    4969 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:06:04.121760 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108723    4969 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:06:04.121760 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108725    4969 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:06:04.121760 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108728    4969 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:06:04.121760 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108730    4969 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:06:04.121760 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108733    4969 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:06:04.122579 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108735    4969 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:06:04.122579 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108740    4969 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:06:04.122579 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108742    4969 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:06:04.122579 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108745    4969 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:06:04.122579 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108747    4969 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:06:04.122579 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108751    4969 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:06:04.122579 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108754    4969 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:06:04.122579 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108756    4969 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:06:04.122579 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108759    4969 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:06:04.122579 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108762    4969 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:06:04.122579 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108765    4969 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:06:04.122579 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108767    4969 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:06:04.122579 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108770    4969 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:06:04.122579 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108773    4969 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:06:04.122579 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108775    4969 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:06:04.122579 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108778    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:06:04.122579 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108781    4969 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:06:04.122579 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108784    4969 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:06:04.122579 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108786    4969 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:06:04.123153 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108788    4969 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:06:04.123153 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108791    4969 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:06:04.123153 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108794    4969 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:06:04.123153 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108796    4969 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:06:04.123153 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108799    4969 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:06:04.123153 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108801    4969 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:06:04.123153 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108804    4969 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:06:04.123153 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108807    4969 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:06:04.123153 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108810    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:06:04.123153 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108813    4969 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:06:04.123153 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108815    4969 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:06:04.123153 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108818    4969 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:06:04.123153 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108821    4969 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:06:04.123153 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108823    4969 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:06:04.123153 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108829    4969 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:06:04.123153 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108832    4969 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:06:04.123153 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108837    4969 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:06:04.123153 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108841    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:06:04.123153 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108844    4969 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:06:04.123611 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108848    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:06:04.123611 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108851    4969 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:06:04.123611 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108853    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:06:04.123611 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108858    4969 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:06:04.123611 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108860    4969 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:06:04.123611 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108863    4969 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:06:04.123611 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108866    4969 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:06:04.123611 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108869    4969 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:06:04.123611 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108872    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:06:04.123611 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108876    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:06:04.123611 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108879    4969 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:06:04.123611 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108881    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:06:04.123611 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108884    4969 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:06:04.123611 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108887    4969 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:06:04.123611 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108889    4969 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:06:04.123611 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108892    4969 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:06:04.123611 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108895    4969 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:06:04.123611 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108897    4969 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:06:04.123611 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108900    4969 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:06:04.124077 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108903    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:06:04.124077 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108905    4969 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:06:04.124077 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108908    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:06:04.124077 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108911    4969 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:06:04.124077 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108913    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:06:04.124077 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108916    4969 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:06:04.124077 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108919    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:06:04.124077 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108921    4969 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:06:04.124077 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108925    4969 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:06:04.124077 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108928    4969 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:06:04.124077 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108930    4969 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:06:04.124077 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108933    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:06:04.124077 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108935    4969 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:06:04.124077 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108938    4969 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:06:04.124077 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108941    4969 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:06:04.124077 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108944    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:06:04.124077 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108947    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:06:04.124077 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108950    4969 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:06:04.124077 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108953    4969 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:06:04.124572 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108955    4969 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:06:04.124572 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108958    4969 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:06:04.124572 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108961    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:06:04.124572 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108963    4969 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:06:04.124572 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108966    4969 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:06:04.124572 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108968    4969 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:06:04.124572 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.108972    4969 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:06:04.124572 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.112977    4969 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 04 02:06:04.124572 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.112991    4969 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 04 02:06:04.124572 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.113009    4969 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 04 02:06:04.124572 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.113014    4969 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 04 02:06:04.124572 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113084    4969 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:06:04.124572 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113104    4969 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:06:04.124572 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113108    4969 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:06:04.124572 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113111    4969 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:06:04.124572 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113114    4969 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:06:04.124572 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113117    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:06:04.124572 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113119    4969 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:06:04.124572 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113122    4969 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:06:04.125032 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113126    4969 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:06:04.125032 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113128    4969 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:06:04.125032 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113131    4969 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:06:04.125032 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113134    4969 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:06:04.125032 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113137    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:06:04.125032 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113140    4969 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:06:04.125032 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113142    4969 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:06:04.125032 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113145    4969 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:06:04.125032 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113155    4969 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:06:04.125032 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113159    4969 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:06:04.125032 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113163    4969 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:06:04.125032 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113166    4969 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:06:04.125032 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113169    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:06:04.125032 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113172    4969 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:06:04.125032 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113174    4969 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:06:04.125032 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113177    4969 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:06:04.125032 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113180    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:06:04.125032 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113182    4969 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:06:04.125032 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113185    4969 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:06:04.125543 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113188    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:06:04.125543 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113190    4969 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:06:04.125543 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113193    4969 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:06:04.125543 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113195    4969 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:06:04.125543 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113198    4969 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:06:04.125543 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113200    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:06:04.125543 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113203    4969 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:06:04.125543 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113205    4969 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:06:04.125543 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113208    4969 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:06:04.125543 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113211    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:06:04.125543 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113213    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:06:04.125543 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113216    4969 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:06:04.125543 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113219    4969 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:06:04.125543 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113222    4969 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:06:04.125543 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113224    4969 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:06:04.125543 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113227    4969 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:06:04.125543 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113230    4969 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:06:04.125543 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113232    4969 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:06:04.125543 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113235    4969 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:06:04.125543 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113237    4969 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:06:04.126040 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113239    4969 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:06:04.126040 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113242    4969 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:06:04.126040 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113250    4969 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:06:04.126040 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113253    4969 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:06:04.126040 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113256    4969 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:06:04.126040 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113259    4969 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:06:04.126040 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113262    4969 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:06:04.126040 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113264    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:06:04.126040 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113267    4969 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:06:04.126040 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113269    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:06:04.126040 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113272    4969 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:06:04.126040 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113275    4969 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:06:04.126040 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113277    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:06:04.126040 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113280    4969 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:06:04.126040 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113282    4969 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:06:04.126040 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113285    4969 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:06:04.126040 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113287    4969 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:06:04.126040 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113290    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:06:04.126040 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113293    4969 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:06:04.126501 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113296    4969 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:06:04.126501 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113300    4969 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:06:04.126501 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113303    4969 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:06:04.126501 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113306    4969 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:06:04.126501 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113308    4969 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:06:04.126501 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113311    4969 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:06:04.126501 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113314    4969 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:06:04.126501 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113316    4969 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:06:04.126501 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113318    4969 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:06:04.126501 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113321    4969 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:06:04.126501 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113324    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:06:04.126501 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113326    4969 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:06:04.126501 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113329    4969 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:06:04.126501 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113332    4969 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:06:04.126501 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113334    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:06:04.126501 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113337    4969 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:06:04.126501 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113349    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:06:04.126501 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113352    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:06:04.126501 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113355    4969 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:06:04.126956 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113358    4969 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:06:04.126956 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113361    4969 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:06:04.126956 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113363    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:06:04.126956 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113366    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:06:04.126956 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113368    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:06:04.126956 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113371    4969 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:06:04.126956 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113373    4969 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:06:04.126956 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113376    4969 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:06:04.126956 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113379    4969 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:06:04.126956 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113381    4969 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:06:04.126956 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113384    4969 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:06:04.126956 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113387    4969 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:06:04.126956 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113389    4969 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:06:04.126956 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113391    4969 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:06:04.126956 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113394    4969 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:06:04.126956 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113397    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:06:04.126956 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113399    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:06:04.126956 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113402    4969 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:06:04.126956 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113405    4969 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:06:04.126956 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113407    4969 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:06:04.127434 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113410    4969 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:06:04.127434 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.113438    4969 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 02:06:04.127434 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113628    4969 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:06:04.127434 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113633    4969 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:06:04.127434 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113642    4969 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:06:04.127434 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113646    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:06:04.127434 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113648    4969 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:06:04.127434 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113651    4969 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:06:04.127434 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113654    4969 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:06:04.127434 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113657    4969 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:06:04.127434 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113666    4969 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:06:04.127434 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113669    4969 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:06:04.127434 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113672    4969 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:06:04.127434 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113675    4969 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:06:04.127434 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113678    4969 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:06:04.127434 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113680    4969 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:06:04.127823 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113683    4969 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:06:04.127823 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113686    4969 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:06:04.127823 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113688    4969 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:06:04.127823 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113691    4969 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:06:04.127823 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113693    4969 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:06:04.127823 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113696    4969 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:06:04.127823 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113699    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:06:04.127823 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113702    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:06:04.127823 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113706    4969 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:06:04.127823 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113710    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:06:04.127823 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113713    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:06:04.127823 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113715    4969 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:06:04.127823 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113718    4969 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:06:04.127823 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113720    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:06:04.127823 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113723    4969 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:06:04.127823 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113726    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:06:04.127823 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113729    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:06:04.127823 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113731    4969 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:06:04.127823 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113734    4969 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:06:04.128378 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113736    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:06:04.128378 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113739    4969 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:06:04.128378 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113741    4969 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:06:04.128378 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113744    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:06:04.128378 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113747    4969 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:06:04.128378 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113749    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:06:04.128378 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113752    4969 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:06:04.128378 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113754    4969 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:06:04.128378 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113762    4969 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:06:04.128378 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113766    4969 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:06:04.128378 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113768    4969 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:06:04.128378 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113771    4969 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:06:04.128378 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113773    4969 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:06:04.128378 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113776    4969 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:06:04.128378 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113779    4969 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:06:04.128378 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113781    4969 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:06:04.128378 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113784    4969 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:06:04.128378 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113787    4969 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:06:04.128378 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113789    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:06:04.128378 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113792    4969 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:06:04.128854 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113794    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:06:04.128854 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113797    4969 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:06:04.128854 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113799    4969 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:06:04.128854 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113802    4969 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:06:04.128854 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113805    4969 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:06:04.128854 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113807    4969 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:06:04.128854 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113810    4969 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:06:04.128854 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113812    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:06:04.128854 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113815    4969 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:06:04.128854 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113817    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:06:04.128854 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113820    4969 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:06:04.128854 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113823    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:06:04.128854 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113825    4969 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:06:04.128854 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113828    4969 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:06:04.128854 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113831    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:06:04.128854 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113834    4969 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:06:04.128854 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113838    4969 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:06:04.128854 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113842    4969 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:06:04.128854 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113845    4969 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:06:04.129399 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113848    4969 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:06:04.129399 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113851    4969 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:06:04.129399 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113859    4969 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:06:04.129399 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113862    4969 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:06:04.129399 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113865    4969 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:06:04.129399 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113868    4969 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:06:04.129399 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113870    4969 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:06:04.129399 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113873    4969 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:06:04.129399 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113875    4969 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:06:04.129399 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113878    4969 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:06:04.129399 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113881    4969 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:06:04.129399 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113883    4969 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:06:04.129399 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113886    4969 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:06:04.129399 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113888    4969 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:06:04.129399 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113891    4969 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:06:04.129399 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113894    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:06:04.129399 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113896    4969 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:06:04.129399 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113899    4969 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:06:04.129399 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113901    4969 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:06:04.129399 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113903    4969 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:06:04.129905 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113906    4969 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:06:04.129905 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113908    4969 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:06:04.129905 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113911    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:06:04.129905 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113914    4969 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:06:04.129905 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113916    4969 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:06:04.129905 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113919    4969 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:06:04.129905 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113922    4969 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:06:04.129905 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113924    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:06:04.129905 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113927    4969 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:06:04.129905 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113929    4969 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:06:04.129905 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113932    4969 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:06:04.129905 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113935    4969 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:06:04.129905 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113937    4969 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:06:04.129905 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.113940    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:06:04.129905 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.113963    4969 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 02:06:04.129905 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114013    4969 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 02:06:04.130309 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114017    4969 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 02:06:04.130309 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114020    4969 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 02:06:04.130309 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114022    4969 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 02:06:04.130309 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114025    4969 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 02:06:04.130309 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114028    4969 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 02:06:04.130309 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114030    4969 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 02:06:04.130309 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114033    4969 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 02:06:04.130309 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114036    4969 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 02:06:04.130309 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114038    4969 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 02:06:04.130309 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114041    4969 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 02:06:04.130309 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114044    4969 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 02:06:04.130309 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114046    4969 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 02:06:04.130309 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114049    4969 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 02:06:04.130309 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114052    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 02:06:04.130309 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114054    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 02:06:04.130309 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114059    4969 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 02:06:04.130309 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114062    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 02:06:04.130309 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114065    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 02:06:04.130309 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114068    4969 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 02:06:04.130788 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114071    4969 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 02:06:04.130788 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114074    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 02:06:04.130788 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114076    4969 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 02:06:04.130788 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114079    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 02:06:04.130788 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114081    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 02:06:04.130788 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114084    4969 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 02:06:04.130788 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114086    4969 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 02:06:04.130788 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114102    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 02:06:04.130788 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114106    4969 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 02:06:04.130788 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114110    4969 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 02:06:04.130788 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114114    4969 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 02:06:04.130788 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114116    4969 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 02:06:04.130788 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114119    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 02:06:04.130788 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114129    4969 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 02:06:04.130788 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114132    4969 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 02:06:04.130788 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114135    4969 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 02:06:04.130788 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114137    4969 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 02:06:04.130788 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114140    4969 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 02:06:04.130788 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114142    4969 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 02:06:04.131294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114145    4969 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 02:06:04.131294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114147    4969 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 02:06:04.131294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114150    4969 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 02:06:04.131294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114153    4969 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 02:06:04.131294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114155    4969 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 02:06:04.131294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114158    4969 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 02:06:04.131294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114160    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 02:06:04.131294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114163    4969 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 02:06:04.131294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114166    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 02:06:04.131294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114168    4969 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 02:06:04.131294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114170    4969 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 02:06:04.131294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114173    4969 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 02:06:04.131294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114176    4969 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 02:06:04.131294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114178    4969 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 02:06:04.131294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114181    4969 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 02:06:04.131294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114183    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 02:06:04.131294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114186    4969 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 02:06:04.131294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114188    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 02:06:04.131294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114208    4969 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 02:06:04.131294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114212    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 02:06:04.131797 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114215    4969 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 02:06:04.131797 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114217    4969 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 02:06:04.131797 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114220    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 02:06:04.131797 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114224    4969 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 02:06:04.131797 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114228    4969 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 02:06:04.131797 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114235    4969 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 02:06:04.131797 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114238    4969 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 02:06:04.131797 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114247    4969 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 02:06:04.131797 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114250    4969 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 02:06:04.131797 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114253    4969 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 02:06:04.131797 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114256    4969 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 02:06:04.131797 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114260    4969 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 02:06:04.131797 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114263    4969 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 02:06:04.131797 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114266    4969 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 02:06:04.131797 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114269    4969 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 02:06:04.131797 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114271    4969 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 02:06:04.131797 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114274    4969 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 02:06:04.131797 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114277    4969 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 02:06:04.131797 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114279    4969 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 02:06:04.131797 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114282    4969 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 02:06:04.132294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114285    4969 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 02:06:04.132294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114287    4969 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 02:06:04.132294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114290    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 02:06:04.132294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114293    4969 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 02:06:04.132294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114296    4969 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 02:06:04.132294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114299    4969 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 02:06:04.132294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114301    4969 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 02:06:04.132294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114304    4969 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 02:06:04.132294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114307    4969 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 02:06:04.132294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114310    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 02:06:04.132294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114313    4969 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 02:06:04.132294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114315    4969 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 02:06:04.132294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114318    4969 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 02:06:04.132294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114320    4969 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 02:06:04.132294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114323    4969 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 02:06:04.132294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114325    4969 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 02:06:04.132294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114328    4969 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 02:06:04.132294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114333    4969 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 02:06:04.132294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114336    4969 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 02:06:04.132294 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114339    4969 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 02:06:04.132762 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114347    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 02:06:04.132762 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114350    4969 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 02:06:04.132762 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114352    4969 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 02:06:04.132762 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114355    4969 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 02:06:04.132762 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114358    4969 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 02:06:04.132762 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114362    4969 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 02:06:04.132762 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:04.114365    4969 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 02:06:04.132762 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.114411    4969 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 04 02:06:04.132762 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.115390    4969 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 04 02:06:04.132762 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.115437    4969 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 04 02:06:04.132762 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.115639    4969 server.go:1014] "Starting client certificate rotation"
Nov 04 02:06:04.132762 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.115847    4969 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 02:06:04.132762 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.115898    4969 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 02:00:22 +0000 UTC" deadline="2028-04-06 15:42:38.351795966 +0000 UTC"
Nov 04 02:06:04.132762 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.116184    4969 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12469h36m34.23562153s"
Nov 04 02:06:04.132762 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.117774    4969 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 02:06:04.132762 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.119120    4969 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 02:06:04.133282 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.120416    4969 log.go:25] "Validated CRI v1 runtime API"
Nov 04 02:06:04.133282 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.122148    4969 log.go:25] "Validated CRI v1 image API"
Nov 04 02:06:04.133282 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.122896    4969 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 04 02:06:04.133282 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.124751    4969 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 bdbc5f21-69be-4d60-877d-26481302a4ff:/dev/nvme0n1p3 ffcd1bb5-c53e-44fe-a557-229452a7ad15:/dev/nvme0n1p4]
Nov 04 02:06:04.133421 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.124772    4969 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/051003e6c47b203cbf6bcdec1d60dbc6c8426203c1020ef1924f9236b7b7acd8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/051003e6c47b203cbf6bcdec1d60dbc6c8426203c1020ef1924f9236b7b7acd8/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/05121522ad496cc0e0f17bc32485494d97f6ca0c3f3d32d000db77ad5b733c9d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/05121522ad496cc0e0f17bc32485494d97f6ca0c3f3d32d000db77ad5b733c9d/userdata/shm major:0 minor:239 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1364fc7f796bc4973dec36326a38998ec7c014e3020006690e2980334e6c5e57/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1364fc7f796bc4973dec36326a38998ec7c014e3020006690e2980334e6c5e57/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/145d997b6376851c6386ed644d5bad00dbc508be47e7c780d1294892033289ad/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/145d997b6376851c6386ed644d5bad00dbc508be47e7c780d1294892033289ad/userdata/shm major:0 minor:224 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/247a62e09494064bc461faadf45e2862c847e8e9c334172d3cef15deee6e0469/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/247a62e09494064bc461faadf45e2862c847e8e9c334172d3cef15deee6e0469/userdata/shm major:0 minor:213 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2764f36f82f901a7bbe63937dbc8c81acb2694d962f903f1cee7c4450283600b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2764f36f82f901a7bbe63937dbc8c81acb2694d962f903f1cee7c4450283600b/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2910f2ca4280823089ec4bb4ff7dc00bde03f703a0d57668c9b4132ab4e49170/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2910f2ca4280823089ec4bb4ff7dc00bde03f703a0d57668c9b4132ab4e49170/userdata/shm major:0 minor:289 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2b730db703d8beaf1d322054237d8680803ef45ed76a4d8ecc2949f5db116de2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2b730db703d8beaf1d322054237d8680803ef45ed76a4d8ecc2949f5db116de2/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/44d76dd4951701fb6150006d4c061465fed28cc004ba46595fd5bdc9c0fb21bb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/44d76dd4951701fb6150006d4c061465fed28cc004ba46595fd5bdc9c0fb21bb/userdata/shm major:0 minor:93 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5daf3f8bae5fdd2aebb2495080ddbb7fbaa802fb0379fbc89f84bb1191c5cae7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5daf3f8bae5fdd2aebb2495080ddbb7fbaa802fb0379fbc89f84bb1191c5cae7/userdata/shm major:0 minor:222 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/64136c8a4dccf871eac64c818fc3144e56f3395df30849785854fc1b2522e763/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/64136c8a4dccf871eac64c818fc3144e56f3395df30849785854fc1b2522e763/userdata/shm major:0 minor:244 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/67c2a63861bcb1cccf0e3cf5e68e34ba2820092939e28368daf4581454e811fc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/67c2a63861bcb1cccf0e3cf5e68e34ba2820092939e28368daf4581454e811fc/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/687ec26b7041d8e6fda8f2619863e7784224c00106904fb64950ce9f99151adc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/687ec26b7041d8e6fda8f2619863e7784224c00106904fb64950ce9f99151adc/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7261e31a7f50bb3a3dde6c082a7623aea0790369aeb5aa0713fec06d6ac30689/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7261e31a7f50bb3a3dde6c082a7623aea0790369aeb5aa0713fec06d6ac30689/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7fcf54c6679939d1561da051a44d5d1f505a57c41aca974fba797ceed6a8a607/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7fcf54c6679939d1561da051a44d5d1f505a57c41aca974fba797ceed6a8a607/userdata/shm major:0 minor:240 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/951057a68760a213b97aa13d3711a9ed29ed8a6657aba282957df5e18f087b24/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/951057a68760a213b97aa13d3711a9ed29ed8a6657aba282957df5e18f087b24/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bb70b336d8f2a92201bc78704b4f67cf0b024535f11f713cae860d164a359578/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bb70b336d8f2a92201bc78704b4f67cf0b024535f11f713cae860d164a359578/userdata/shm major:0 minor:215 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c0f566aa8fe4d1aa54333085c0b2dda4804f9e81bf730ae858b8411ca2451e5c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c0f566aa8fe4d1aa54333085c0b2dda4804f9e81bf730ae858b8411ca2451e5c/userdata/shm major:0 minor:336 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c6d2b1ba17da682935416cb2e38cf5c0b7197bfa360dc9c158dd133b3fc93a77/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c6d2b1ba17da682935416cb2e38cf5c0b7197bfa360dc9c158dd133b3fc93a77/userdata/shm major:0 minor:346 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c878d0aba7f0a2dd52a414833678ea583c46fb5436076ca931e1c257fa3092f3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c878d0aba7f0a2dd52a414833678ea583c46fb5436076ca931e1c257fa3092f3/userdata/shm major:0 minor:334 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c8cc6d9aa1515773d997e809800d5d88fc1e54d6fcbf91dbc603822b31dac17d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c8cc6d9aa1515773d997e809800d5d88fc1e54d6fcbf91dbc603822b31dac17d/userdata/shm major:0 minor:246 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d2d7df19176a6115af6bea2c4c097c8ae4bbd930b5fc1d1bbab812c714ed3d9e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d2d7df19176a6115af6bea2c4c097c8ae4bbd930b5fc1d1bbab812c714ed3d9e/userdata/shm major:0 minor:231 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d98fb4942db8640f591aad36e7dcd313c88fb76f0789cb58851ffeaf8aa1e0db/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d98fb4942db8640f591aad36e7dcd313c88fb76f0789cb58851ffeaf8aa1e0db/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/dc5dff4d3474af86324f5aa9943a05aa786faf6d3c08346b5025e477d181d7af/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/dc5dff4d3474af86324f5aa9943a05aa786faf6d3c08346b5025e477d181d7af/userdata/shm major:0 minor:288 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e22425b440bc67c66afa0eebbedb50069edb060ce7965835cdab6c3da015f5a1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e22425b440bc67c66afa0eebbedb50069edb060ce7965835cdab6c3da015f5a1/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93/volumes/kubernetes.io~empty-dir/tmp major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93/volumes/kubernetes.io~projected/kube-api-access-9pl9r:{mountpoint:/var/lib/kubelet/pods/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93/volumes/kubernetes.io~projected/kube-api-access-9pl9r major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/12e9dc19-8f06-4b5d-8237-3ed5a449e09f/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/12e9dc19-8f06-4b5d-8237-3ed5a449e09f/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:234 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/12e9dc19-8f06-4b5d-8237-3ed5a449e09f/volumes/kubernetes.io~projected/kube-api-access-l4gp2:{mountpoint:/var/lib/kubelet/pods/12e9dc19-8f06-4b5d-8237-3ed5a449e09f/volumes/kubernetes.io~projected/kube-api-access-l4gp2 major:0 minor:237 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/12e9dc19-8f06-4b5d-8237-3ed5a449e09f/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/12e9dc19-8f06-4b5d-8237-3ed5a449e09f/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/12e9dc19-8f06-4b5d-8237-3ed5a449e09f/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/12e9dc19-8f06-4b5d-8237-3ed5a449e09f/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/150189f8-cd39-4a16-9d07-112edc6e83dc/volumes/kubernetes.io~projected/kube-api-access-j2v2n:{mountpoint:/var/lib/kubelet/pods/150189f8-cd39-4a16-9d07-112edc6e83dc/volumes/kubernetes.io~projected/kube-api-access-j2v2n major:0 minor:236 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/150189f8-cd39-4a16-9d07-112edc6e83dc/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/150189f8-cd39-4a16-9d07-112edc6e83dc/volumes/kubernetes.io~secret/serving-cert major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1597f8a8-5b13-4076-9d95-c494cf3ad5dc/volumes/kubernetes.io~projected/kube-api-access-nqvtx:{mountpoint:/var/lib/kubelet/pods/1597f8a8-5b13-4076-9d95-c494cf3ad5dc/volumes/kubernetes.io~projected/kube-api-access-nqvtx major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/16c89b3c-f63f-4489-8eae-5918677e1e61/volumes/kubernetes.io~projected/kube-api-access-85d9k:{mountpoint:/var/lib/kubelet/pods/16c89b3c-f63f-4489-8eae-5918677e1e61/volumes/kubernetes.io~projected/kube-api-access-85d9k major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/16f00163-6ec3-4c10-81e4-c8a501e15e82/volumes/kubernetes.io~projected/kube-api-access-96kfp:{mountpoint:/var/lib/kubelet/pods/16f00163-6ec3-4c10-81e4-c8a501e15e82/volumes/kubernetes.io~projected/kube-api-access-96kfp major:0 minor:235 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/16f00163-6ec3-4c10-81e4-c8a501e15e82/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/16f00163-6ec3-4c10-81e4-c8a501e15e82/volumes/kubernetes.io~secret/serving-cert major:0 minor:228 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/301f5fdf-8bd3-49b3-be41-5684c210577c/volumes/kubernetes.io~projected/kube-api-access-k8st8:{mountpoint:/var/lib/kubelet/pods/301f5fdf-8bd3-49b3-be41-5684c210577c/volumes/kubernetes.io~projected/kube-api-access-k8st8 major:0 minor:233 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/301f5fdf-8bd3-49b3-be41-5684c210577c/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/301f5fdf-8bd3-49b3-be41-5684c210577c/volumes/kubernetes.io~secret/cert major:0 minor:230 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/32699b90-bd7d-43f5-bdcc-53de9da6f244/volumes/kubernetes.io~projected/kube-api-access-qcfqp:{mountpoint:/var/lib/kubelet/pods/32699b90-bd7d-43f5-bdcc-53de9da6f244/volumes/kubernetes.io~projected/kube-api-access-qcfqp major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/32699b90-bd7d-43f5-bdcc-53de9da6f244/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/32699b90-bd7d-43f5-bdcc-53de9da6f244/volumes/kubernetes.io~secret/default-certificate major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/32699b90-bd7d-43f5-bdcc-53de9da6f244/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/32699b90-bd7d-43f5-bdcc-53de9da6f244/volumes/kubernetes.io~secret/stats-auth major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/37d0fd53-4873-4d8e-9472-6754d577c9a5/volumes/kubernetes.io~projected/kube-api-access-mrwzs:{mountpoint:/var/lib/kubelet/pods/37d0fd53-4873-4d8e-9472-6754d577c9a5/volumes/kubernetes.io~projected/kube-api-access-mrwzs major:0 minor:238 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/39235bb9-36b4-43e4-9161-043a3fb6a261/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/39235bb9-36b4-43e4-9161-043a3fb6a261/volumes/kubernetes.io~secret/agent-certs major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3f0365d3-076c-4128-9349-b7dd71181e5f/volumes/kubernetes.io~projected/kube-api-access-98v7l:{mountpoint:/var/lib/kubelet/pods/3f0365d3-076c-4128-9349-b7dd71181e5f/volumes/kubernetes.io~projected/kube-api-access-98v7l major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/430bb96c-bf05-4a46-bdd3-612bd19a26d8/volumes/kubernetes.io~projected/kube-api-access-6vg7p:{mountpoint:/var/lib/kubelet/pods/430bb96c-bf05-4a46-bdd3-612bd19a26d8/volumes/kubernetes.io~projected/kube-api-access-6vg7p major:0 minor:333 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d/volumes/kubernetes.io~projected/kube-api-access-l89wj:{mountpoint:/var/lib/kubelet/pods/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d/volumes/kubernetes.io~projected/kube-api-access-l89wj major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/63254ae3-0393-4094-9401-e18067d30080/volumes/kubernetes.io~projected/kube-api-access-rvghh:{mountpoint:/var/lib/kubelet/pods/63254ae3-0393-4094-9401-e18067d30080/volumes/kubernetes.io~projected/kube-api-access-rvghh major:0 minor:243 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa/volumes/kubernetes.io~projected/kube-api-access-8vb2x:{mountpoint:/var/lib/kubelet/pods/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa/volumes/kubernetes.io~projected/kube-api-access-8vb2x major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/70aa0113-0423-4bdf-8335-d6c5bd707757/volumes/kubernetes.io~projected/kube-api-access-fcv2t:{mountpoint:/var/lib/kubelet/pods/70aa0113-0423-4bdf-8335-d6c5bd707757/volumes/kubernetes.io~projected/kube-api-access-fcv2t major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7c365e54-8c98-4271-a66e-9b13fe91afa7/volumes/kubernetes.io~projected/kube-api-access-s9j8z:{mountpoint:/var/lib/kubelet/pods/7c365e54-8c98-4271-a66e-9b13fe91afa7/volumes/kubernetes.io~projected/kube-api-access-s9j8z major:0 minor:316 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7cdf1fc0-00ea-4108-8ff0-5c031a193968/volumes/kubernetes.io~projected/kube-api-access-cnpf6:{mountpoint:/var/lib/kubelet/pods/7cdf1fc0-00ea-4108-8ff0-5c031a193968/volumes/kubernetes.io~projected/kube-api-access-cnpf6 major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7cdf1fc0-00ea-4108-8ff0-5c031a193968/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/7cdf1fc0-00ea-4108-8ff0-5c031a193968/volumes/kubernetes.io~secret/serving-cert major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8ddd2cf3-5cce-41a8-ad26-6946d3cb5910/volumes/kubernetes.io~projected/kube-api-access-lcxzt:{mountpoint:/var/lib/kubelet/pods/8ddd2cf3-5cce-41a8-ad26-6946d3cb5910/volumes/kubernetes.io~projected/kube-api-access-lcxzt major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8ec1a73f-2254-499d-ab62-96c8c223ed8d/volumes/kubernetes.io~projected/kube-api-access-kjm7q:{mountpoint:/var/lib/kubelet/pods/8ec1a73f-2254-499d-ab62-96c8c223ed8d/volumes/kubernetes.io~projected/kube-api-access-kjm7q major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/94fedaf4-9e63-4c96-bb3e-e508815dd6c2/volumes/kubernetes.io~projected/kube-api-access-482t2:{mountpoint:/var/lib/kubelet/pods/94fedaf4-9e63-4c96-bb3e-e508815dd6c2/volumes/kubernetes.io~projected/kube-api-access-482t2 major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9a395e26-3eac-4728-bbad-c42eaf6da56d/volumes/kubernetes.io~projected/kube-api-access-84hdg:{mountpoint:/var/lib/kubelet/pods/9a395e26-3eac-4728-bbad-c42eaf6da56d/volumes/kubernetes.io~projected/kube-api-access-84hdg major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d/volumes/kubernetes.io~projected/kube-api-access-x24q6:{mountpoint:/var/lib/kubelet/pods/a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d/volumes/kubernetes.io~projected/kube-api-access-x24q6 major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d/volumes/kubernetes.io~secret/serving-cert major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/abdfaf94-b721-4ed4-8671-416e694df5bb/volumes/kubernetes.io~projected/kube-api-access-l6l5g:{mountpoint:/var/lib/kubelet/pods/abdfaf94-b721-4ed4-8671-416e694df5bb/volumes/kubernetes.io~projected/kube-api-access-l6l5g major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/afe4b3d8-a18c-4587-9057-83123ffe3347/volumes/kubernetes.io~projected/kube-api-access-fr84q:{mountpoint:/var/lib/kubelet/pods/afe4b3d8-a18c-4587-9057-83123ffe3347/volumes/kubernetes.io~projected/kube-api-access-fr84q major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b0a72217-fe1d-4589-9201-651503694cb4/volumes/kubernetes.io~projected/kube-api-access-nrkqj:{mountpoint:/var/lib/kubelet/pods/b0a72217-fe1d-4589-9201-651503694cb4/volumes/kubernetes.io~projected/kube-api-access-nrkqj major:0 minor:287 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b2156825-c0d8-48a4-a90f-2340e98a13e3/volumes/kubernetes.io~projected/kube-api-access-b7f7m:{mountpoint:/var/lib/kubelet/pods/b2156825-c0d8-48a4-a90f-2340e98a13e3/volumes/kubernetes.io~projected/kube-api-access-b7f7m major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9/volumes/kubernetes.io~projected/kube-api-access-gn4d2:{mountpoint:/var/lib/kubelet/pods/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9/volumes/kubernetes.io~projected/kube-api-access-gn4d2 major:0 minor:338 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cab3ee15-5d68-4840-83f4-ea8e2308e530/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/cab3ee15-5d68-4840-83f4-ea8e2308e530/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cab3ee15-5d68-4840-83f4-ea8e2308e530/volumes/kubernetes.io~projected/kube-api-access-h77nn:{mountpoint:/var/lib/kubelet/pods/cab3ee15-5d68-4840-83f4-ea8e2308e530/volumes/kubernetes.io~projected/kube-api-access-h77nn major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cab3ee15-5d68-4840-83f4-ea8e2308e530/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/cab3ee15-5d68-4840-83f4-ea8e2308e530/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cab3ee15-5d68-4840-83f4-ea8e2308e530/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/cab3ee15-5d68-4840-83f4-ea8e2308e530/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1/volumes/kubernetes.io~projected/kube-api-access-hkch2:{mountpoint:/var/lib/kubelet/pods/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1/volumes/kubernetes.io~projected/kube-api-access-hkch2 major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e164905d-5505-4d20-af2e-b8acd370bf18/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/e164905d-5505-4d20-af2e-b8acd370bf18/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:286 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eb9e13cc-14be-46c7-bea5-2501b0a85486/volumes/kubernetes.io~projected/kube-api-access-qm6tq:{mountpoint:/var/lib/kubelet/pods/eb9e13cc-14be-46c7-bea5-2501b0a85486/volumes/kubernetes.io~projected/kube-api-access-qm6tq major:0 minor:345 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/f56b403ca651790e425359545914895a17f2ab06412d1fb4f797502f41deb1eb/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/6b269e8f44e8261384e695924f1973ef56025ef6e1756443fba69c747d826f82/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/4d6b6e60881485fc6c4f49eeab22a577f25fb8c40e74819e5a05636b98b039f0/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/d78e8a5f9b6c21a79454a16fbb55ca9285cbe35f3730c1be31dbb112d23f1d7b/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/3d7eaf73748a07498c9308fdf04e09529328868e13ce4ad8045857de8d3101fa/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/5e49bc245a959336f412c8854aaea239757acbb7f52644f79afaf86527d7d7bc/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-121:{mountpoint:/var/lib/containers/storage/overlay/d278b14d6405b016f8ea458273442be3669a30caa2a71cfa75ba4794fda99086/merged major:0 minor:121 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/093938bf81d9f4e791805269414927e9046aeb66592045cdde2401261e70a74f/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-129:{mountpoint:/var/lib/containers/storage/overlay/d2414b3ba5cfbd4a7dee86170479f930a2edd0465fbfcc7d0ba5245f8d8cc0fd/merged major:0 minor:129 fsType:overlay blockSize:0} overlay_0-131:{mountpoint:/var/lib/containers/storage/overlay/61d99eb59a58545c370b9851b5cf4f36e131555535fa0898ef6f5dec268042b9/merged major:0 minor:131 fsType:overlay blockSize:0} overlay_0-135:{mountpoint:/var/lib/containers/storage/overlay/7850e21ec55d113bceac741b2d9eb03e11f02de31dbf6644754b1936fc45d0b2/merged major:0 minor:135 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/31eb2a81dfdef793e2bdc1682d6470c5b5b91b3482ce79610f8c8faca8a13476/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-151:{mountpoint:/var/lib/containers/storage/overlay/4c3980ca8cd8aa486798a9fd1140ace5dcb93bbb943821135c543e2c5efe2d5e/merged major:0 minor:151 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/15d686be7b6ed9227571949094af40fe4035836dd080c4967d4b3ff8e946c0a1/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/b043d5639c63938f1751257f55c5d1f532df6911ce622f91f9e2d1ba0d71b764/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/58cfa6304dbc647a085d04833b87e8abe861e6cea1658e5b5791036565b0d89a/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/c31f6f53b83e45d07c51d3a5aa25ee16ac7058632720893108b6c0f39f3cac0b/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/b8ef9c3e3ca1bb8f6c52409edad63decf4e40bc93e44590e86208b05b43f92e9/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/00077547692e1ab3ccb5ce1cca48fa8b0058673c8a8f09569fed067b6041b6a6/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/39bd3d61f729c49071f50b01cb44f3c748d335d848105af733acfa9c07f93c90/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/6566ffd942ab63fda332f0d23097029180657bd8644c8c724ab943a58eff72c9/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/9e16c1814e81f6307a974385f89e37f3d0ede33e1ad30bcfc7c5308b5f7e7e49/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/ff3ea2d14dc8924129f70c483b23767a3654fe3611ecd22d59842dda104184ec/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/c1ed950ea5423d529ffd3defae8e20867eba1657427990501e6dfd961feccb06/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/b265a2a498d7f96dd3f8482a1378f4584b9c5464be7584037d99b4535db35f05/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/0f7cad6ee12a7db19e01ee90a58e81625cfd78ce974379b3cf3b41cbc86d13b1/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/76860e1b9622b56bb13334486746374b4e65864032c56c6b6ad79bba6adce13a/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/3a45c7cb8ecf5453b0d2e03a6f8c2397f1386ab6400e3240c71f61086c6fdbe3/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/785cbe9b5177d6a2ad0a6807d82d4ef358b35a2e4f735bb2eda05fa390c69430/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/0fd99d90c2051e34240359c8d9f4923c4d0e27c96b18147914af7477281885ad/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-266:{mountpoint:/var/lib/containers/storage/overlay/dab57dd3b79392ae8b0dce6739914e234ebfd73c66d6aa0e3df22b8db644ae3b/merged major:0 minor:266 fsType:overlay blockSize:0} overlay_0-267:{mountpoint:/var/lib/containers/storage/overlay/62c26f654d86a4d81f8308c1059c874d5790d049f998d74d02d41e4e30e8fed9/merged major:0 minor:267 fsType:overlay blockSize:0} overlay_0-282:{mountpoint:/var/lib/containers/storage/overlay/0f33d91ddbfefe988a3dc48d5a31223db84b473d47a1623c9df5422cf46c7756/merged major:0 minor:282 fsType:overlay blockSize:0} overlay_0-284:{mountpoint:/var/lib/containers/storage/overlay/21c9c0421575dd099c1a5be74ce73b4a6862a2af06c7cf124450db91a2bf163d/merged major:0 minor:284 fsType:overlay blockSize:0} overlay_0-292:{mountpoint:/var/lib/containers/storage/overlay/589c0128f0d19a469691bc9ccda61043eb5a7bef6dbb9dcb4930db2f71c41ba2/merged major:0 minor:292 fsType:overlay blockSize:0} overlay_0-294:{mountpoint:/var/lib/containers/storage/overlay/de68296975343527ff72a5ac4279ced47799d20096c2eeaa0b385989878f9845/merged major:0 minor:294 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/2ec5f0bcd16c937e33d937f9022cadaeca3452e81dde0c43d0ce60ec31cf1aa9/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/76a7c8ea4e445347fc7e50833c256f88af53774c90f6554a9499378df1c23bb1/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/0c5908245f72f97998cb98b6400351b0b618c921cdbe5ee5a9052ebe16cb71bd/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-310:{mountpoint:/var/lib/containers/storage/overlay/5967811ee716fc29ee961b075b0bb4ad9dc87f021d4e8a26237bb99cbbd3b4ba/merged major:0 minor:310 fsType:overlay blockSize:0} overlay_0-312:{mountpoint:/var/lib/containers/storage/overlay/3bea4937f379b99f5ffb6b99852e49a528b2bf4085b85cc89c77445d8f4ec6f9/merged major:0 minor:312 fsType:overlay blockSize:0} overlay_0-314:{mountpoint:/var/lib/containers/storage/overlay/09e4f0ded637b4e05c7725e4d3fb953954b71bfa3dc04423f4bfef92dcac36a4/merged major:0 minor:314 fsType:overlay blockSize:0} overlay_0-339:{mountpoint:/var/lib/containers/storage/overlay/70919a69a8f09fe6533dad2211d8df9bfa6146c31d6d41fcb150691ff2da358e/merged major:0 minor:339 fsType:overlay blockSize:0} overlay_0-340:{mountpoint:/var/lib/containers/storage/overlay/7865dd7a7201d1dcc8d76d0b51bf1569923862de8e5218b746c9c7deabd6b94d/merged major:0 minor:340 fsType:overlay blockSize:0} overlay_0-343:{mountpoint:/var/lib/containers/storage/overlay/baab75a2c840e96497266ababdd246d89d2103b0251ae169a6bb0d30ec910f3c/merged major:0 minor:343 fsType:overlay blockSize:0} overlay_0-352:{mountpoint:/var/lib/containers/storage/overlay/93c6c26c428a714bb1391fe683a022823a8c6930233aefbd8bf7c4b30e4db248/merged major:0 minor:352 fsType:overlay blockSize:0} overlay_0-354:{mountpoint:/var/lib/containers/storage/overlay/7aafa4133ff9ca85cbd8a4dae202531bcda71a9124ed785b2e0fee4bf57b899b/merged major:0 minor:354 fsType:overlay blockSize:0} overlay_0-359:{mountpoint:/var/lib/containers/storage/overlay/230522e93e9919575074dae712c5e971602a840da0055522e3fe260c813e8954/merged major:0 minor:359 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/fb72836ba94d73c75e723de49bb44e0c9e74839992a3617ed1980522abe1133d/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/d8d129f8ee018f906c2dc05fc232b74cab2d730f1849aa6a67dfafbfd09d6106/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/b8df1423b2c544d8f678f10ad603f61d609e67441e7c4b82b03b53e73a3d2f16/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/81c20a48c4f5d5b53ca545419bff7a0500783e173821a7966813200b9e9603b2/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/aac4794234b74aaa4f3fb17c06f5e1f9171e5da96c14aa5d52a8f48fbfd4efac/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/6dd7520b6128f0e3a5b9873cdb3850086a05196d215dd2e4c285dcb479c59ea3/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/dacc51a6f989a71e90bbfe42bc2c83ed2ba1ea6275e37d03632cc833f66b7f6e/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/58d329a739bd3e4484e93f56df5bce12b022f6eebf8302711e2a2235739814bf/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/6935cf47475012b118a9291b6244975bd049e9cd819f8bc5145bad1a7642ae70/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/e0303976637fdedf492feb872a29500f8d9bbecf6c21b2a2ccd35f5f78af902a/merged major:0 minor:96 fsType:overlay blockSize:0}]
Nov 04 02:06:04.143026 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.142664    4969 manager.go:217] Machine: {Timestamp:2026-11-04 02:06:04.142210555 +0000 UTC m=+0.068433178 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098999 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22b7e5da8137204d2fc9ce09009a9f SystemUUID:ec22b7e5-da81-3720-4d2f-c9ce09009a9f BootID:93c774cc-f5a6-4440-aa40-e5480a1c729b Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-129 DeviceMajor:0 DeviceMinor:129 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d98fb4942db8640f591aad36e7dcd313c88fb76f0789cb58851ffeaf8aa1e0db/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/32699b90-bd7d-43f5-bdcc-53de9da6f244/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:197 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-284 DeviceMajor:0 DeviceMinor:284 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/32699b90-bd7d-43f5-bdcc-53de9da6f244/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:201 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:209 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5daf3f8bae5fdd2aebb2495080ddbb7fbaa802fb0379fbc89f84bb1191c5cae7/userdata/shm DeviceMajor:0 DeviceMinor:222 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2764f36f82f901a7bbe63937dbc8c81acb2694d962f903f1cee7c4450283600b/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-131 DeviceMajor:0 DeviceMinor:131 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c6d2b1ba17da682935416cb2e38cf5c0b7197bfa360dc9c158dd133b3fc93a77/userdata/shm DeviceMajor:0 DeviceMinor:346 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/16c89b3c-f63f-4489-8eae-5918677e1e61/volumes/kubernetes.io~projected/kube-api-access-85d9k DeviceMajor:0 DeviceMinor:62 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cab3ee15-5d68-4840-83f4-ea8e2308e530/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:211 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93/volumes/kubernetes.io~projected/kube-api-access-9pl9r DeviceMajor:0 DeviceMinor:66 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-121 DeviceMajor:0 DeviceMinor:121 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/abdfaf94-b721-4ed4-8671-416e694df5bb/volumes/kubernetes.io~projected/kube-api-access-l6l5g DeviceMajor:0 DeviceMinor:45 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/37d0fd53-4873-4d8e-9472-6754d577c9a5/volumes/kubernetes.io~projected/kube-api-access-mrwzs DeviceMajor:0 DeviceMinor:238 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-354 DeviceMajor:0 DeviceMinor:354 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:55 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/afe4b3d8-a18c-4587-9057-83123ffe3347/volumes/kubernetes.io~projected/kube-api-access-fr84q DeviceMajor:0 DeviceMinor:67 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-135 DeviceMajor:0 DeviceMinor:135 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/301f5fdf-8bd3-49b3-be41-5684c210577c/volumes/kubernetes.io~projected/kube-api-access-k8st8 DeviceMajor:0 DeviceMinor:233 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/dc5dff4d3474af86324f5aa9943a05aa786faf6d3c08346b5025e477d181d7af/userdata/shm DeviceMajor:0 DeviceMinor:288 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-292 DeviceMajor:0 DeviceMinor:292 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1364fc7f796bc4973dec36326a38998ec7c014e3020006690e2980334e6c5e57/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7261e31a7f50bb3a3dde6c082a7623aea0790369aeb5aa0713fec06d6ac30689/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9a395e26-3eac-4728-bbad-c42eaf6da56d/volumes/kubernetes.io~projected/kube-api-access-84hdg DeviceMajor:0 DeviceMinor:63 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1/volumes/kubernetes.io~projected/kube-api-access-hkch2 DeviceMajor:0 DeviceMinor:64 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/64136c8a4dccf871eac64c818fc3144e56f3395df30849785854fc1b2522e763/userdata/shm DeviceMajor:0 DeviceMinor:244 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/run/containers/storage/overlay-containers/67c2a63861bcb1cccf0e3cf5e68e34ba2820092939e28368daf4581454e811fc/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-343 DeviceMajor:0 DeviceMinor:343 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/3f0365d3-076c-4128-9349-b7dd71181e5f/volumes/kubernetes.io~projected/kube-api-access-98v7l DeviceMajor:0 DeviceMinor:218 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/12e9dc19-8f06-4b5d-8237-3ed5a449e09f/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:229 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d2d7df19176a6115af6bea2c4c097c8ae4bbd930b5fc1d1bbab812c714ed3d9e/userdata/shm DeviceMajor:0 DeviceMinor:231 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e22425b440bc67c66afa0eebbedb50069edb060ce7965835cdab6c3da015f5a1/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7cdf1fc0-00ea-4108-8ff0-5c031a193968/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:210 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cab3ee15-5d68-4840-83f4-ea8e2308e530/volumes/kubernetes.io~projected/kube-api-access-h77nn DeviceMajor:0 DeviceMinor:220 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/12e9dc19-8f06-4b5d-8237-3ed5a449e09f/volumes/kubernetes.io~projected/kube-api-access-l4gp2 DeviceMajor:0 DeviceMinor:237 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-312 DeviceMajor:0 DeviceMinor:312 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8ec1a73f-2254-499d-ab62-96c8c223ed8d/volumes/kubernetes.io~projected/kube-api-access-kjm7q DeviceMajor:0 DeviceMinor:202 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/247a62e09494064bc461faadf45e2862c847e8e9c334172d3cef15deee6e0469/userdata/shm DeviceMajor:0 DeviceMinor:213 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-339 DeviceMajor:0 DeviceMinor:339 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/94fedaf4-9e63-4c96-bb3e-e508815dd6c2/volumes/kubernetes.io~projected/kube-api-access-482t2 DeviceMajor:0 DeviceMinor:65 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cab3ee15-5d68-4840-83f4-ea8e2308e530/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:212 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c8cc6d9aa1515773d997e809800d5d88fc1e54d6fcbf91dbc603822b31dac17d/userdata/shm DeviceMajor:0 DeviceMinor:246 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e164905d-5505-4d20-af2e-b8acd370bf18/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:286 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8ddd2cf3-5cce-41a8-ad26-6946d3cb5910/volumes/kubernetes.io~projected/kube-api-access-lcxzt DeviceMajor:0 DeviceMinor:60 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/16f00163-6ec3-4c10-81e4-c8a501e15e82/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:228 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/301f5fdf-8bd3-49b3-be41-5684c210577c/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:230 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-359 DeviceMajor:0 DeviceMinor:359 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cab3ee15-5d68-4840-83f4-ea8e2308e530/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:221 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7fcf54c6679939d1561da051a44d5d1f505a57c41aca974fba797ceed6a8a607/userdata/shm DeviceMajor:0 DeviceMinor:240 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-294 DeviceMajor:0 DeviceMinor:294 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:56 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/430bb96c-bf05-4a46-bdd3-612bd19a26d8/volumes/kubernetes.io~projected/kube-api-access-6vg7p DeviceMajor:0 DeviceMinor:333 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c878d0aba7f0a2dd52a414833678ea583c46fb5436076ca931e1c257fa3092f3/userdata/shm DeviceMajor:0 DeviceMinor:334 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa/volumes/kubernetes.io~projected/kube-api-access-8vb2x DeviceMajor:0 DeviceMinor:205 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bb70b336d8f2a92201bc78704b4f67cf0b024535f11f713cae860d164a359578/userdata/shm DeviceMajor:0 DeviceMinor:215 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-267 DeviceMajor:0 DeviceMinor:267 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7c365e54-8c98-4271-a66e-9b13fe91afa7/volumes/kubernetes.io~projected/kube-api-access-s9j8z DeviceMajor:0 DeviceMinor:316 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9/volumes/kubernetes.io~projected/kube-api-access-gn4d2 DeviceMajor:0 DeviceMinor:338 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b2156825-c0d8-48a4-a90f-2340e98a13e3/volumes/kubernetes.io~projected/kube-api-access-b7f7m DeviceMajor:0 DeviceMinor:203 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/145d997b6376851c6386ed644d5bad00dbc508be47e7c780d1294892033289ad/userdata/shm DeviceMajor:0 DeviceMinor:224 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/12e9dc19-8f06-4b5d-8237-3ed5a449e09f/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:234 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-266 DeviceMajor:0 DeviceMinor:266 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d/volumes/kubernetes.io~projected/kube-api-access-x24q6 DeviceMajor:0 DeviceMinor:217 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/150189f8-cd39-4a16-9d07-112edc6e83dc/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:225 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-314 DeviceMajor:0 DeviceMinor:314 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:57 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/32699b90-bd7d-43f5-bdcc-53de9da6f244/volumes/kubernetes.io~projected/kube-api-access-qcfqp DeviceMajor:0 DeviceMinor:204 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/05121522ad496cc0e0f17bc32485494d97f6ca0c3f3d32d000db77ad5b733c9d/userdata/shm DeviceMajor:0 DeviceMinor:239 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/39235bb9-36b4-43e4-9161-043a3fb6a261/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:58 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/44d76dd4951701fb6150006d4c061465fed28cc004ba46595fd5bdc9c0fb21bb/userdata/shm DeviceMajor:0 DeviceMinor:93 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-282 DeviceMajor:0 DeviceMinor:282 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-352 DeviceMajor:0 DeviceMinor:352 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7cdf1fc0-00ea-4108-8ff0-5c031a193968/volumes/kubernetes.io~projected/kube-api-access-cnpf6 DeviceMajor:0 DeviceMinor:219 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/150189f8-cd39-4a16-9d07-112edc6e83dc/volumes/kubernetes.io~projected/kube-api-access-j2v2n DeviceMajor:0 DeviceMinor:236 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2910f2ca4280823089ec4bb4ff7dc00bde03f703a0d57668c9b4132ab4e49170/userdata/shm DeviceMajor:0 DeviceMinor:289 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-151 DeviceMajor:0 DeviceMinor:151 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2b730db703d8beaf1d322054237d8680803ef45ed76a4d8ecc2949f5db116de2/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/63254ae3-0393-4094-9401-e18067d30080/volumes/kubernetes.io~projected/kube-api-access-rvghh DeviceMajor:0 DeviceMinor:243 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c0f566aa8fe4d1aa54333085c0b2dda4804f9e81bf730ae858b8411ca2451e5c/userdata/shm DeviceMajor:0 DeviceMinor:336 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-340 DeviceMajor:0 DeviceMinor:340 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/70aa0113-0423-4bdf-8335-d6c5bd707757/volumes/kubernetes.io~projected/kube-api-access-fcv2t DeviceMajor:0 DeviceMinor:59 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/951057a68760a213b97aa13d3711a9ed29ed8a6657aba282957df5e18f087b24/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/12e9dc19-8f06-4b5d-8237-3ed5a449e09f/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:226 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-310 DeviceMajor:0 DeviceMinor:310 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/eb9e13cc-14be-46c7-bea5-2501b0a85486/volumes/kubernetes.io~projected/kube-api-access-qm6tq DeviceMajor:0 DeviceMinor:345 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/687ec26b7041d8e6fda8f2619863e7784224c00106904fb64950ce9f99151adc/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d/volumes/kubernetes.io~projected/kube-api-access-l89wj DeviceMajor:0 DeviceMinor:61 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b0a72217-fe1d-4589-9201-651503694cb4/volumes/kubernetes.io~projected/kube-api-access-nrkqj DeviceMajor:0 DeviceMinor:287 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/16f00163-6ec3-4c10-81e4-c8a501e15e82/volumes/kubernetes.io~projected/kube-api-access-96kfp DeviceMajor:0 DeviceMinor:235 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/051003e6c47b203cbf6bcdec1d60dbc6c8426203c1020ef1924f9236b7b7acd8/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1597f8a8-5b13-4076-9d95-c494cf3ad5dc/volumes/kubernetes.io~projected/kube-api-access-nqvtx DeviceMajor:0 DeviceMinor:206 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:05121522ad496cc MacAddress:52:e1:91:49:70:2e Speed:10000 Mtu:8901} {Name:145d997b6376851 MacAddress:32:fa:a0:bd:c3:a0 Speed:10000 Mtu:8901} {Name:247a62e09494064 MacAddress:4e:e1:c6:74:e1:b0 Speed:10000 Mtu:8901} {Name:2910f2ca4280823 MacAddress:4a:9a:09:58:05:6c Speed:10000 Mtu:8901} {Name:2b730db703d8bea MacAddress:ba:07:66:8a:ce:97 Speed:10000 Mtu:8901} {Name:5daf3f8bae5fdd2 MacAddress:ae:25:2c:ce:34:82 Speed:10000 Mtu:8901} {Name:64136c8a4dccf87 MacAddress:ee:b7:ea:e5:7b:39 Speed:10000 Mtu:8901} {Name:7fcf54c6679939d MacAddress:0a:90:d8:24:ea:a6 Speed:10000 Mtu:8901} {Name:bb70b336d8f2a92 MacAddress:4a:a1:81:75:fa:9e Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:46:67:df:df:cf Speed:0 Mtu:9001} {Name:br-int MacAddress:12:0a:7b:b7:ad:60 Speed:0 Mtu:8901} {Name:c0f566aa8fe4d1a MacAddress:2a:9b:f6:29:da:38 Speed:10000 Mtu:8901} {Name:c6d2b1ba17da682 MacAddress:3e:05:41:95:fb:55 Speed:10000 Mtu:8901} {Name:c878d0aba7f0a2d MacAddress:22:e6:bb:4a:f8:ff Speed:10000 Mtu:8901} {Name:c8cc6d9aa151577 MacAddress:c6:f9:e8:f7:01:ca Speed:10000 Mtu:8901} {Name:d2d7df19176a611 MacAddress:12:5b:0f:4b:0e:7b Speed:10000 Mtu:8901} {Name:dc5dff4d3474af8 MacAddress:52:8e:00:bb:8c:59 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:46:67:df:df:cf Speed:0 Mtu:9001} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:aa:86:bb:8b:a1:11 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 04 02:06:04.143277 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.143032    4969 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 04 02:06:04.143277 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.143076    4969 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 04 02:06:04.143397 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.143379    4969 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 04 02:06:04.143536 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.143398    4969 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-63.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 04 02:06:04.143577 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.143537    4969 topology_manager.go:143] "Creating topology manager with none policy"
Nov 04 02:06:04.143577 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.143545    4969 container_manager_linux.go:311] "Creating device plugin manager"
Nov 04 02:06:04.143577 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.143558    4969 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 02:06:04.143664 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.143586    4969 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 02:06:04.143692 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.143669    4969 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 04 02:06:04.143864 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.143855    4969 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 04 02:06:04.143994 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.143986    4969 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 04 02:06:04.144029 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.144021    4969 kubelet.go:493] "Attempting to sync node with API server"
Nov 04 02:06:04.144057 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.144035    4969 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 04 02:06:04.144057 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.144044    4969 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 04 02:06:04.144057 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.144053    4969 kubelet.go:405] "Adding apiserver pod source"
Nov 04 02:06:04.144176 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.144061    4969 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 04 02:06:04.145888 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.145377    4969 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 04 02:06:04.145888 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.145399    4969 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 02:06:04.146037 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.146018    4969 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 04 02:06:04.157389 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.157374    4969 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 04 02:06:04.157389 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.157389    4969 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 02:06:04.157498 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.157491    4969 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 04 02:06:04.157692 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.157680    4969 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 04 02:06:04.157722 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.157710    4969 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 04 02:06:04.157818 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.157806    4969 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 04 02:06:04.157851 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.157827    4969 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 04 02:06:04.157851 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.157839    4969 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 04 02:06:04.157851 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.157846    4969 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 04 02:06:04.157945 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.157852    4969 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 04 02:06:04.157945 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.157858    4969 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 04 02:06:04.157945 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.157863    4969 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 04 02:06:04.157945 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.157869    4969 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 04 02:06:04.157945 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.157876    4969 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 04 02:06:04.157945 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.157881    4969 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 04 02:06:04.157945 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.157890    4969 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 04 02:06:04.157945 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.157903    4969 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 04 02:06:04.157945 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.157929    4969 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 04 02:06:04.157945 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.157939    4969 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 04 02:06:04.161400 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.161386    4969 server.go:1263] "Started kubelet"
Nov 04 02:06:04.161501 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.161465    4969 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 04 02:06:04.161553 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.161484    4969 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 04 02:06:04.161601 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.161582    4969 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 04 02:06:04.162008 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.161990    4969 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 04 02:06:04.162115 ip-10-0-136-63 systemd[1]: Started Kubernetes Kubelet.
Nov 04 02:06:04.163206 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.162759    4969 server.go:317] "Adding debug handlers to kubelet server"
Nov 04 02:06:04.165019 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.164929    4969 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 04 02:06:04.165446 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.165408    4969 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 04 02:06:04.165718 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.165632    4969 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 02:00:22 +0000 UTC" deadline="2028-06-08 05:36:25.568933831 +0000 UTC"
Nov 04 02:06:04.165718 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.165669    4969 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13971h30m21.403268672s"
Nov 04 02:06:04.166212 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.165813    4969 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 04 02:06:04.166212 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.165826    4969 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 04 02:06:04.166212 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.165891    4969 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 04 02:06:04.166212 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:04.166060    4969 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-63.us-west-1.compute.internal\" not found"
Nov 04 02:06:04.170987 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.170968    4969 factory.go:55] Registering systemd factory
Nov 04 02:06:04.170987 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.170988    4969 factory.go:223] Registration of the systemd container factory successfully
Nov 04 02:06:04.171200 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171043    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cab3ee15-5d68-4840-83f4-ea8e2308e530" volumeName="kubernetes.io/secret/cab3ee15-5d68-4840-83f4-ea8e2308e530-installation-pull-secrets" seLinuxMountContext=""
Nov 04 02:06:04.171200 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171079    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="12e9dc19-8f06-4b5d-8237-3ed5a449e09f" volumeName="kubernetes.io/secret/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-image-registry-private-configuration" seLinuxMountContext=""
Nov 04 02:06:04.171200 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171118    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="430bb96c-bf05-4a46-bdd3-612bd19a26d8" volumeName="kubernetes.io/projected/430bb96c-bf05-4a46-bdd3-612bd19a26d8-kube-api-access-6vg7p" seLinuxMountContext=""
Nov 04 02:06:04.171200 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171132    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63254ae3-0393-4094-9401-e18067d30080" volumeName="kubernetes.io/empty-dir/63254ae3-0393-4094-9401-e18067d30080-tmp-dir" seLinuxMountContext=""
Nov 04 02:06:04.171200 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171143    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7cdf1fc0-00ea-4108-8ff0-5c031a193968" volumeName="kubernetes.io/empty-dir/7cdf1fc0-00ea-4108-8ff0-5c031a193968-snapshots" seLinuxMountContext=""
Nov 04 02:06:04.171200 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171157    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d" volumeName="kubernetes.io/configmap/a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d-config" seLinuxMountContext=""
Nov 04 02:06:04.171200 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171169    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb9e13cc-14be-46c7-bea5-2501b0a85486" volumeName="kubernetes.io/empty-dir/eb9e13cc-14be-46c7-bea5-2501b0a85486-archives-path" seLinuxMountContext=""
Nov 04 02:06:04.171200 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171182    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="078e0741-51f6-4ae3-b9c0-bdf82a0f8f93" volumeName="kubernetes.io/empty-dir/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-tmp" seLinuxMountContext=""
Nov 04 02:06:04.171200 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171197    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9a395e26-3eac-4728-bbad-c42eaf6da56d" volumeName="kubernetes.io/configmap/9a395e26-3eac-4728-bbad-c42eaf6da56d-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171208    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c41ceadb-e895-4b54-b3ea-88e0b6aa56d9" volumeName="kubernetes.io/empty-dir/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-data-volume" seLinuxMountContext=""
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171220    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="12e9dc19-8f06-4b5d-8237-3ed5a449e09f" volumeName="kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-bound-sa-token" seLinuxMountContext=""
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171232    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="150189f8-cd39-4a16-9d07-112edc6e83dc" volumeName="kubernetes.io/configmap/150189f8-cd39-4a16-9d07-112edc6e83dc-config" seLinuxMountContext=""
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171242    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="16f00163-6ec3-4c10-81e4-c8a501e15e82" volumeName="kubernetes.io/secret/16f00163-6ec3-4c10-81e4-c8a501e15e82-serving-cert" seLinuxMountContext=""
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171250    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="39235bb9-36b4-43e4-9161-043a3fb6a261" volumeName="kubernetes.io/configmap/39235bb9-36b4-43e4-9161-043a3fb6a261-konnectivity-ca" seLinuxMountContext=""
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171260    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="12e9dc19-8f06-4b5d-8237-3ed5a449e09f" volumeName="kubernetes.io/secret/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-installation-pull-secrets" seLinuxMountContext=""
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171267    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3f0365d3-076c-4128-9349-b7dd71181e5f" volumeName="kubernetes.io/configmap/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-config" seLinuxMountContext=""
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171274    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7cdf1fc0-00ea-4108-8ff0-5c031a193968" volumeName="kubernetes.io/projected/7cdf1fc0-00ea-4108-8ff0-5c031a193968-kube-api-access-cnpf6" seLinuxMountContext=""
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171285    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c41ceadb-e895-4b54-b3ea-88e0b6aa56d9" volumeName="kubernetes.io/projected/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-kube-api-access-gn4d2" seLinuxMountContext=""
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171297    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cab3ee15-5d68-4840-83f4-ea8e2308e530" volumeName="kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-bound-sa-token" seLinuxMountContext=""
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171305    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cab3ee15-5d68-4840-83f4-ea8e2308e530" volumeName="kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-kube-api-access-h77nn" seLinuxMountContext=""
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171312    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1597f8a8-5b13-4076-9d95-c494cf3ad5dc" volumeName="kubernetes.io/projected/1597f8a8-5b13-4076-9d95-c494cf3ad5dc-kube-api-access-nqvtx" seLinuxMountContext=""
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171319    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="16c89b3c-f63f-4489-8eae-5918677e1e61" volumeName="kubernetes.io/configmap/16c89b3c-f63f-4489-8eae-5918677e1e61-iptables-alerter-script" seLinuxMountContext=""
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171327    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d" volumeName="kubernetes.io/projected/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-kube-api-access-l89wj" seLinuxMountContext=""
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171335    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="70aa0113-0423-4bdf-8335-d6c5bd707757" volumeName="kubernetes.io/empty-dir/70aa0113-0423-4bdf-8335-d6c5bd707757-tmp-dir" seLinuxMountContext=""
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171342    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b2156825-c0d8-48a4-a90f-2340e98a13e3" volumeName="kubernetes.io/projected/b2156825-c0d8-48a4-a90f-2340e98a13e3-kube-api-access-b7f7m" seLinuxMountContext=""
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171350    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa" volumeName="kubernetes.io/configmap/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-telemetry-config" seLinuxMountContext=""
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171370    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="70aa0113-0423-4bdf-8335-d6c5bd707757" volumeName="kubernetes.io/projected/70aa0113-0423-4bdf-8335-d6c5bd707757-kube-api-access-fcv2t" seLinuxMountContext=""
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171377    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="078e0741-51f6-4ae3-b9c0-bdf82a0f8f93" volumeName="kubernetes.io/empty-dir/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-tuned" seLinuxMountContext=""
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171390    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0851f8eb-225e-4576-b5bc-dc6d72eefb6c" volumeName="kubernetes.io/configmap/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-nginx-conf" seLinuxMountContext=""
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171398    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="301f5fdf-8bd3-49b3-be41-5684c210577c" volumeName="kubernetes.io/secret/301f5fdf-8bd3-49b3-be41-5684c210577c-cert" seLinuxMountContext=""
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171405    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1" volumeName="kubernetes.io/secret/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171422    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7cdf1fc0-00ea-4108-8ff0-5c031a193968" volumeName="kubernetes.io/empty-dir/7cdf1fc0-00ea-4108-8ff0-5c031a193968-tmp" seLinuxMountContext=""
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171440    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8ec1a73f-2254-499d-ab62-96c8c223ed8d" volumeName="kubernetes.io/projected/8ec1a73f-2254-499d-ab62-96c8c223ed8d-kube-api-access-kjm7q" seLinuxMountContext=""
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171447    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9a395e26-3eac-4728-bbad-c42eaf6da56d" volumeName="kubernetes.io/configmap/9a395e26-3eac-4728-bbad-c42eaf6da56d-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171455    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3f0365d3-076c-4128-9349-b7dd71181e5f" volumeName="kubernetes.io/projected/3f0365d3-076c-4128-9349-b7dd71181e5f-kube-api-access-98v7l" seLinuxMountContext=""
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171461    4969 factory.go:153] Registering CRI-O factory
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171462    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d" volumeName="kubernetes.io/configmap/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-multus-daemon-config" seLinuxMountContext=""
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171476    4969 factory.go:223] Registration of the crio container factory successfully
Nov 04 02:06:04.171466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171477    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63254ae3-0393-4094-9401-e18067d30080" volumeName="kubernetes.io/configmap/63254ae3-0393-4094-9401-e18067d30080-config-volume" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171486    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d" volumeName="kubernetes.io/secret/a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d-serving-cert" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171493    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cab3ee15-5d68-4840-83f4-ea8e2308e530" volumeName="kubernetes.io/configmap/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-certificates" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171501    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb9e13cc-14be-46c7-bea5-2501b0a85486" volumeName="kubernetes.io/configmap/eb9e13cc-14be-46c7-bea5-2501b0a85486-service-ca-bundle" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171508    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="12e9dc19-8f06-4b5d-8237-3ed5a449e09f" volumeName="kubernetes.io/configmap/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-trusted-ca" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171516    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="12e9dc19-8f06-4b5d-8237-3ed5a449e09f" volumeName="kubernetes.io/empty-dir/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-ca-trust-extracted" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171524    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="37d0fd53-4873-4d8e-9472-6754d577c9a5" volumeName="kubernetes.io/projected/37d0fd53-4873-4d8e-9472-6754d577c9a5-kube-api-access-mrwzs" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171531    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="39235bb9-36b4-43e4-9161-043a3fb6a261" volumeName="kubernetes.io/secret/39235bb9-36b4-43e4-9161-043a3fb6a261-agent-certs" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171538    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="12e9dc19-8f06-4b5d-8237-3ed5a449e09f" volumeName="kubernetes.io/configmap/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-certificates" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171547    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7cdf1fc0-00ea-4108-8ff0-5c031a193968" volumeName="kubernetes.io/configmap/7cdf1fc0-00ea-4108-8ff0-5c031a193968-service-ca-bundle" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171554    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="afe4b3d8-a18c-4587-9057-83123ffe3347" volumeName="kubernetes.io/projected/afe4b3d8-a18c-4587-9057-83123ffe3347-kube-api-access-fr84q" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171562    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c41ceadb-e895-4b54-b3ea-88e0b6aa56d9" volumeName="kubernetes.io/configmap/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-kube-rbac-proxy-cm" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171570    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cab3ee15-5d68-4840-83f4-ea8e2308e530" volumeName="kubernetes.io/empty-dir/cab3ee15-5d68-4840-83f4-ea8e2308e530-ca-trust-extracted" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171577    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa" volumeName="kubernetes.io/projected/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-kube-api-access-8vb2x" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171592    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1" volumeName="kubernetes.io/configmap/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-ovnkube-script-lib" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171600    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="301f5fdf-8bd3-49b3-be41-5684c210577c" volumeName="kubernetes.io/projected/301f5fdf-8bd3-49b3-be41-5684c210577c-kube-api-access-k8st8" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171608    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="32699b90-bd7d-43f5-bdcc-53de9da6f244" volumeName="kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-stats-auth" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171616    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7cdf1fc0-00ea-4108-8ff0-5c031a193968" volumeName="kubernetes.io/configmap/7cdf1fc0-00ea-4108-8ff0-5c031a193968-trusted-ca-bundle" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171624    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94fedaf4-9e63-4c96-bb3e-e508815dd6c2" volumeName="kubernetes.io/projected/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-kube-api-access-482t2" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171631    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b0a72217-fe1d-4589-9201-651503694cb4" volumeName="kubernetes.io/projected/b0a72217-fe1d-4589-9201-651503694cb4-kube-api-access-nrkqj" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171644    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1" volumeName="kubernetes.io/configmap/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-ovnkube-config" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171651    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="12e9dc19-8f06-4b5d-8237-3ed5a449e09f" volumeName="kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-kube-api-access-l4gp2" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171658    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="32699b90-bd7d-43f5-bdcc-53de9da6f244" volumeName="kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-default-certificate" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171666    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9a395e26-3eac-4728-bbad-c42eaf6da56d" volumeName="kubernetes.io/configmap/9a395e26-3eac-4728-bbad-c42eaf6da56d-cni-binary-copy" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171673    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="abdfaf94-b721-4ed4-8671-416e694df5bb" volumeName="kubernetes.io/projected/abdfaf94-b721-4ed4-8671-416e694df5bb-kube-api-access-l6l5g" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171680    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e164905d-5505-4d20-af2e-b8acd370bf18" volumeName="kubernetes.io/secret/e164905d-5505-4d20-af2e-b8acd370bf18-original-pull-secret" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171687    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb9e13cc-14be-46c7-bea5-2501b0a85486" volumeName="kubernetes.io/projected/eb9e13cc-14be-46c7-bea5-2501b0a85486-kube-api-access-qm6tq" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171694    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7cdf1fc0-00ea-4108-8ff0-5c031a193968" volumeName="kubernetes.io/secret/7cdf1fc0-00ea-4108-8ff0-5c031a193968-serving-cert" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171518    4969 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171725    4969 factory.go:103] Registering Raw factory
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171741    4969 manager.go:1199] Started watching for new ooms in manager
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171701    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8ddd2cf3-5cce-41a8-ad26-6946d3cb5910" volumeName="kubernetes.io/projected/8ddd2cf3-5cce-41a8-ad26-6946d3cb5910-kube-api-access-lcxzt" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171778    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="32699b90-bd7d-43f5-bdcc-53de9da6f244" volumeName="kubernetes.io/projected/32699b90-bd7d-43f5-bdcc-53de9da6f244-kube-api-access-qcfqp" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171785    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="078e0741-51f6-4ae3-b9c0-bdf82a0f8f93" volumeName="kubernetes.io/projected/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-kube-api-access-9pl9r" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171792    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="16f00163-6ec3-4c10-81e4-c8a501e15e82" volumeName="kubernetes.io/configmap/16f00163-6ec3-4c10-81e4-c8a501e15e82-config" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171799    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="16f00163-6ec3-4c10-81e4-c8a501e15e82" volumeName="kubernetes.io/projected/16f00163-6ec3-4c10-81e4-c8a501e15e82-kube-api-access-96kfp" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171806    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d" volumeName="kubernetes.io/configmap/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-cni-binary-copy" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171815    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1" volumeName="kubernetes.io/projected/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-kube-api-access-hkch2" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171823    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="150189f8-cd39-4a16-9d07-112edc6e83dc" volumeName="kubernetes.io/secret/150189f8-cd39-4a16-9d07-112edc6e83dc-serving-cert" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171830    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="16f00163-6ec3-4c10-81e4-c8a501e15e82" volumeName="kubernetes.io/configmap/16f00163-6ec3-4c10-81e4-c8a501e15e82-trusted-ca" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171837    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63254ae3-0393-4094-9401-e18067d30080" volumeName="kubernetes.io/projected/63254ae3-0393-4094-9401-e18067d30080-kube-api-access-rvghh" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171845    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9a395e26-3eac-4728-bbad-c42eaf6da56d" volumeName="kubernetes.io/projected/9a395e26-3eac-4728-bbad-c42eaf6da56d-kube-api-access-84hdg" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171852    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d" volumeName="kubernetes.io/projected/a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d-kube-api-access-x24q6" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171859    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1" volumeName="kubernetes.io/configmap/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-env-overrides" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171873    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eb9e13cc-14be-46c7-bea5-2501b0a85486" volumeName="kubernetes.io/configmap/eb9e13cc-14be-46c7-bea5-2501b0a85486-trusted-ca-bundle" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171880    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="150189f8-cd39-4a16-9d07-112edc6e83dc" volumeName="kubernetes.io/projected/150189f8-cd39-4a16-9d07-112edc6e83dc-kube-api-access-j2v2n" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171888    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="16c89b3c-f63f-4489-8eae-5918677e1e61" volumeName="kubernetes.io/projected/16c89b3c-f63f-4489-8eae-5918677e1e61-kube-api-access-85d9k" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171896    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c365e54-8c98-4271-a66e-9b13fe91afa7" volumeName="kubernetes.io/projected/7c365e54-8c98-4271-a66e-9b13fe91afa7-kube-api-access-s9j8z" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171902    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8ddd2cf3-5cce-41a8-ad26-6946d3cb5910" volumeName="kubernetes.io/configmap/8ddd2cf3-5cce-41a8-ad26-6946d3cb5910-serviceca" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171909    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cab3ee15-5d68-4840-83f4-ea8e2308e530" volumeName="kubernetes.io/configmap/cab3ee15-5d68-4840-83f4-ea8e2308e530-trusted-ca" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171918    4969 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cab3ee15-5d68-4840-83f4-ea8e2308e530" volumeName="kubernetes.io/secret/cab3ee15-5d68-4840-83f4-ea8e2308e530-image-registry-private-configuration" seLinuxMountContext=""
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171926    4969 reconstruct.go:97] "Volume reconstruction finished"
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.171931    4969 reconciler.go:29] "Reconciler: start to sync state"
Nov 04 02:06:04.172398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.172075    4969 manager.go:319] Starting recovery of all containers
Nov 04 02:06:04.174302 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.172896    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jcqmp_16f00163-6ec3-4c10-81e4-c8a501e15e82/console-operator/1.log"
Nov 04 02:06:04.180909 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.176052    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jcqmp_16f00163-6ec3-4c10-81e4-c8a501e15e82/console-operator/1.log"
Nov 04 02:06:04.192998 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.192968    4969 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 04 02:06:04.194235 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.194216    4969 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 04 02:06:04.194331 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.194254    4969 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 04 02:06:04.194331 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.194274    4969 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 04 02:06:04.194877 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:04.194320    4969 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 04 02:06:04.205753 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.205726    4969 generic.go:356] "Generic (PLEG): container finished" podID="9a395e26-3eac-4728-bbad-c42eaf6da56d" containerID="24b3e751dae85c639a2e9dbbc671c48b63c8e32a5823fce96b9d539cc61d02e4" exitCode=0
Nov 04 02:06:04.205753 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.205752    4969 generic.go:356] "Generic (PLEG): container finished" podID="9a395e26-3eac-4728-bbad-c42eaf6da56d" containerID="6771cfcfe3192d6b89a907f38fd0cb2de2de3ca1a3f2dba8e14e72e47a1152da" exitCode=0
Nov 04 02:06:04.205908 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.205759    4969 generic.go:356] "Generic (PLEG): container finished" podID="9a395e26-3eac-4728-bbad-c42eaf6da56d" containerID="98b80cdcbc81b9d6ecda8c882280361794cc76236bcdc7ba5e6e8909527abeee" exitCode=0
Nov 04 02:06:04.205908 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.205768    4969 generic.go:356] "Generic (PLEG): container finished" podID="9a395e26-3eac-4728-bbad-c42eaf6da56d" containerID="a88fa3c2d03d6f89de2a0adb1c815dce46b3a160a6266127d10ea7705216c756" exitCode=0
Nov 04 02:06:04.205908 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.205776    4969 generic.go:356] "Generic (PLEG): container finished" podID="9a395e26-3eac-4728-bbad-c42eaf6da56d" containerID="7c47f05c740a168a51556d758528a767ca77c1618027af7dc423a98707963ae6" exitCode=0
Nov 04 02:06:04.205908 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.205783    4969 generic.go:356] "Generic (PLEG): container finished" podID="9a395e26-3eac-4728-bbad-c42eaf6da56d" containerID="7ab82a1249d696be0410a2bf34b49dd839f561e2a7568a1b7297abfe9a0084f1" exitCode=0
Nov 04 02:06:04.213528 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.213497    4969 generic.go:356] "Generic (PLEG): container finished" podID="1ec35563e4041daa397c9c3ffc626b1c" containerID="aa72b53adf7911d7c704e045a8bd15aa7130a7ca3660c6470d64bd4074e87ae0" exitCode=0
Nov 04 02:06:04.244323 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.244272    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jcqmp_16f00163-6ec3-4c10-81e4-c8a501e15e82/console-operator/1.log"
Nov 04 02:06:04.244323 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.244309    4969 generic.go:356] "Generic (PLEG): container finished" podID="16f00163-6ec3-4c10-81e4-c8a501e15e82" containerID="54ef74383bad59d1840a87bd68172bf3587b9c10deb740b68015579e57aeebae" exitCode=255
Nov 04 02:06:04.265875 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.265855    4969 manager.go:324] Recovery completed
Nov 04 02:06:04.266180 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:04.266162    4969 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-63.us-west-1.compute.internal\" not found"
Nov 04 02:06:04.283628 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.283611    4969 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 02:06:04.284738 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.284723    4969 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 02:06:04.284815 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.284755    4969 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 02:06:04.284815 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.284770    4969 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 02:06:04.287223 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.287206    4969 cpu_manager.go:226] "Starting" policy="none"
Nov 04 02:06:04.287223 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.287218    4969 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 02:06:04.287310 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.287221    4969 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 04 02:06:04.287310 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.287307    4969 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 04 02:06:04.287427 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.287414    4969 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 04 02:06:04.287457 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.287428    4969 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 04 02:06:04.287457 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.287445    4969 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 04 02:06:04.287457 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.287450    4969 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 04 02:06:04.287457 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.287455    4969 policy_none.go:50] "Start"
Nov 04 02:06:04.287566 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.287463    4969 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 04 02:06:04.287566 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.287473    4969 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 04 02:06:04.287630 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.287603    4969 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 04 02:06:04.287630 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.287612    4969 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 04 02:06:04.287630 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.287617    4969 policy_none.go:44] "Start"
Nov 04 02:06:04.288112 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.288087    4969 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 02:06:04.288151 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.288124    4969 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 02:06:04.288151 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.288134    4969 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 02:06:04.292163 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.291988    4969 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 04 02:06:04.294395 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:04.294365    4969 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Nov 04 02:06:04.298368 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.298353    4969 manager.go:341] "Starting Device Plugin manager"
Nov 04 02:06:04.298432 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:04.298382    4969 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 04 02:06:04.298432 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.298393    4969 server.go:87] "Starting device plugin registration server"
Nov 04 02:06:04.298640 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.298627    4969 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 04 02:06:04.298681 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.298640    4969 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 04 02:06:04.298749 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.298733    4969 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 04 02:06:04.298913 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.298879    4969 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 04 02:06:04.299055 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.299038    4969 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 04 02:06:04.299055 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.299052    4969 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 04 02:06:04.299228 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.299115    4969 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T02:06:04.299026629Z","UUID":"e5885891-f579-47b3-8de2-5d5faa509242","Handler":null,"Name":"","Endpoint":""}
Nov 04 02:06:04.300239 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:04.300219    4969 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 04 02:06:04.300319 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:04.300256    4969 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-63.us-west-1.compute.internal\" not found"
Nov 04 02:06:04.300319 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.300282    4969 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 04 02:06:04.300319 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.300311    4969 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 04 02:06:04.353400 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.353372    4969 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 02:06:04.353513 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.353498    4969 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 04 02:06:04.399717 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.399698    4969 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 02:06:04.400657 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.400637    4969 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 04 02:06:04.400742 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.400671    4969 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 02:06:04.400742 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.400685    4969 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 04 02:06:04.400742 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.400736    4969 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:06:04.403747 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.403729    4969 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 02:06:04.403875 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.403860    4969 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 02:06:04.404030 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.404017    4969 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 02:06:04.495175 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.495110    4969 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal"]
Nov 04 02:06:04.495789 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.495759    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal" event={"ID":"1ec35563e4041daa397c9c3ffc626b1c","Type":"ContainerStarted","Data":"f1fa2effa39f93d396b21fbef0c5f5a734494d27146f8b2e4550d6b6f28c1960"}
Nov 04 02:06:04.495840 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.495790    4969 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:06:04.495840 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.495798    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal" event={"ID":"1ec35563e4041daa397c9c3ffc626b1c","Type":"ContainerDied","Data":"aa72b53adf7911d7c704e045a8bd15aa7130a7ca3660c6470d64bd4074e87ae0"}
Nov 04 02:06:04.495840 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.495813    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal" event={"ID":"1ec35563e4041daa397c9c3ffc626b1c","Type":"ContainerStarted","Data":"687ec26b7041d8e6fda8f2619863e7784224c00106904fb64950ce9f99151adc"}
Nov 04 02:06:04.495925 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.495857    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal" event={"ID":"5dd9721b3a00168bde79d6cad3060599","Type":"ContainerStarted","Data":"a28dc15040456383218742865fd3d0a93eebbe2a0a4f754868fabb0ea9750afb"}
Nov 04 02:06:04.495925 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.495874    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal" event={"ID":"5dd9721b3a00168bde79d6cad3060599","Type":"ContainerStarted","Data":"1364fc7f796bc4973dec36326a38998ec7c014e3020006690e2980334e6c5e57"}
Nov 04 02:06:04.496018 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.496004    4969 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:06:04.509750 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.509731    4969 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:06:04.509823 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.509783    4969 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:06:04.510250 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.510231    4969 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeNotReady"
Nov 04 02:06:04.510293 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.510271    4969 setters.go:546] "Node became not ready" node="ip-10-0-136-63.us-west-1.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-11-04T02:06:04Z","lastTransitionTime":"2026-11-04T02:06:04Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"}
Nov 04 02:06:04.516492 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.516474    4969 nodeinfomanager.go:471] existing CSINode "ip-10-0-136-63.us-west-1.compute.internal" is owned by different node (oldNodeID="cae206de-1195-4f26-ac9b-d883dbc68b67", newNodeID=""), cleaning up...
Nov 04 02:06:04.559318 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.559295    4969 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 02:06:04.559398 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:04.559334    4969 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:06:04.562915 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.562899    4969 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 02:06:04.562982 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:04.562935    4969 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:06:04.655760 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.655735    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5dd9721b3a00168bde79d6cad3060599-config\") pod \"kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal\" (UID: \"5dd9721b3a00168bde79d6cad3060599\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:06:04.655870 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.655764    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1ec35563e4041daa397c9c3ffc626b1c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal\" (UID: \"1ec35563e4041daa397c9c3ffc626b1c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:06:04.655870 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.655781    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ec35563e4041daa397c9c3ffc626b1c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal\" (UID: \"1ec35563e4041daa397c9c3ffc626b1c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:06:04.756495 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.756435    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5dd9721b3a00168bde79d6cad3060599-config\") pod \"kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal\" (UID: \"5dd9721b3a00168bde79d6cad3060599\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:06:04.756495 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.756466    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1ec35563e4041daa397c9c3ffc626b1c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal\" (UID: \"1ec35563e4041daa397c9c3ffc626b1c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:06:04.756495 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.756483    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ec35563e4041daa397c9c3ffc626b1c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal\" (UID: \"1ec35563e4041daa397c9c3ffc626b1c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:06:04.756662 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.756522    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ec35563e4041daa397c9c3ffc626b1c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal\" (UID: \"1ec35563e4041daa397c9c3ffc626b1c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:06:04.756662 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.756541    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5dd9721b3a00168bde79d6cad3060599-config\") pod \"kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal\" (UID: \"5dd9721b3a00168bde79d6cad3060599\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:06:04.756662 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:04.756556    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1ec35563e4041daa397c9c3ffc626b1c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal\" (UID: \"1ec35563e4041daa397c9c3ffc626b1c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:06:05.144279 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.144248    4969 apiserver.go:52] "Watching apiserver"
Nov 04 02:06:05.254497 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.254469    4969 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:06:05.255007 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.254989    4969 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:06:05.310543 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.310522    4969 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 04 02:06:05.311254 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.311223    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td","openshift-image-registry/node-ca-gzg65","kube-system/global-pull-secret-syncer-b4wjr","openshift-service-ca/service-ca-5fffc97bdf-pwwpf","openshift-ovn-kubernetes/ovnkube-node-2l29t","openshift-multus/multus-additional-cni-plugins-jsh5m","kube-system/kas-connection-checker-685895c7c9-pn5gm","openshift-ingress-canary/ingress-canary-gflvk","openshift-network-diagnostics/network-check-source-6f78d59f99-62blg","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal","openshift-cluster-node-tuning-operator/tuned-98zkj","openshift-image-registry/image-registry-5bd9d74c6b-thtzz","openshift-multus/multus-hclbr","openshift-multus/network-metrics-daemon-4v7xm","openshift-dns/node-resolver-vjp7p","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-96g6r","openshift-network-diagnostics/network-check-target-98h6z","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xxtnw","openshift-console-operator/console-operator-7cd56b8dbc-jcqmp","kube-system/kas-connection-checker-685895c7c9-2mklg","openshift-network-console/networking-console-plugin-86c478bb88-pxvln","openshift-image-registry/image-registry-6bc84fd8c9-wbbgn","openshift-insights/insights-operator-7c4986d449-6cc2t","openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg","openshift-kube-storage-version-migrator/migrator-66df5d496b-dcf2j","openshift-kube-storage-version-migrator/migrator-66df5d496b-nvl6s","openshift-network-operator/iptables-alerter-qvk5m","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb","openshift-service-ca-operator/service-ca-operator-544d749c8c-c2vfk","kube-system/kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal","openshift-insights/insights-runtime-extractor-hh6m5","openshift-dns/dns-default-8sr5s","kube-system/konnectivity-agent-928nm","openshift-insights/periodic-gathering-xwzqv-7rqm2","kube-system/kas-connection-checker-685895c7c9-d27bw","openshift-ingress/router-default-6595f8bf85-dmj26"]
Nov 04 02:06:05.312561 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.312539    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:06:05.313162 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.313144    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg"
Nov 04 02:06:05.313880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.313860    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:06:05.314303 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.314282    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"
Nov 04 02:06:05.314839 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.314817    4969 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 02:06:05.314925 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:05.314895    4969 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:06:05.316415 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.316398    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-pxvln"
Nov 04 02:06:05.317933 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.317905    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:06:05.318246 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.318230    4969 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 02:06:05.318315 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:05.318266    4969 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-63.us-west-1.compute.internal"
Nov 04 02:06:05.324161 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.324138    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:06:05.324852 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.324840    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:06:05.330483 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.330465    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:05.333434 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.333418    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-pwwpf"
Nov 04 02:06:05.365055 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.365034    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Nov 04 02:06:05.365055 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.365036    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Nov 04 02:06:05.365421 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.365407    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 04 02:06:05.365475 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.365409    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 04 02:06:05.365475 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.365411    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Nov 04 02:06:05.365723 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.365709    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 04 02:06:05.365774 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.365757    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 04 02:06:05.366483 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.366468    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 04 02:06:05.366524 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.366501    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 04 02:06:05.367375 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.367359    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 04 02:06:05.367624 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.367607    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Nov 04 02:06:05.367754 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.367739    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 04 02:06:05.367833 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.367817    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Nov 04 02:06:05.367876 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.367848    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 04 02:06:05.367876 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.367854    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 04 02:06:05.367991 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.367973    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Nov 04 02:06:05.368055 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.368010    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Nov 04 02:06:05.368055 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.368020    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 04 02:06:05.368055 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.368027    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Nov 04 02:06:05.368176 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.368074    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Nov 04 02:06:05.368176 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.368128    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Nov 04 02:06:05.368349 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.368333    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 04 02:06:05.368413 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.368400    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Nov 04 02:06:05.368601 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.368572    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 04 02:06:05.368793 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.368776    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Nov 04 02:06:05.368879 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.368795    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Nov 04 02:06:05.368879 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.368798    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Nov 04 02:06:05.368879 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.368831    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-lb5t8\""
Nov 04 02:06:05.368879 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.368841    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Nov 04 02:06:05.368879 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.368832    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-jnl5r\""
Nov 04 02:06:05.368879 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.368851    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Nov 04 02:06:05.369066 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.368939    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Nov 04 02:06:05.369886 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.369872    4969 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 04 02:06:05.414498 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.414445    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-tdnwm\""
Nov 04 02:06:05.414498 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.414458    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Nov 04 02:06:05.414681 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.414664    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-n8gzm\""
Nov 04 02:06:05.414720 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.414678    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-dvtjz\""
Nov 04 02:06:05.414875 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.414859    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wbgvz\""
Nov 04 02:06:05.414947 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.414936    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-nwfcm\""
Nov 04 02:06:05.415278 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.415264    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 04 02:06:05.416835 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.416819    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Nov 04 02:06:05.417125 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.417112    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 04 02:06:05.460878 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.460855    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-var-lib-openvswitch\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.460953 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.460885    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-data-volume\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:05.460953 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.460905    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb9e13cc-14be-46c7-bea5-2501b0a85486-trusted-ca-bundle\") pod \"periodic-gathering-xwzqv-7rqm2\" (UID: \"eb9e13cc-14be-46c7-bea5-2501b0a85486\") " pod="openshift-insights/periodic-gathering-xwzqv-7rqm2"
Nov 04 02:06:05.460953 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.460930    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-h72td\" (UID: \"3f0365d3-076c-4128-9349-b7dd71181e5f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"
Nov 04 02:06:05.461054 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.460988    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98v7l\" (UniqueName: \"kubernetes.io/projected/3f0365d3-076c-4128-9349-b7dd71181e5f-kube-api-access-98v7l\") pod \"cluster-samples-operator-6ddb6c8b94-h72td\" (UID: \"3f0365d3-076c-4128-9349-b7dd71181e5f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"
Nov 04 02:06:05.461085 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.461062    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs\") pod \"network-metrics-daemon-4v7xm\" (UID: \"afe4b3d8-a18c-4587-9057-83123ffe3347\") " pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:06:05.461152 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.461135    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-data-volume\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:05.461192 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.461143    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-var-lib-cni-bin\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.461223 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.461187    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-certificates\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:06:05.461316 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.461297    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-ovnkube-script-lib\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.461375 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.461312    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb9e13cc-14be-46c7-bea5-2501b0a85486-trusted-ca-bundle\") pod \"periodic-gathering-xwzqv-7rqm2\" (UID: \"eb9e13cc-14be-46c7-bea5-2501b0a85486\") " pod="openshift-insights/periodic-gathering-xwzqv-7rqm2"
Nov 04 02:06:05.461375 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.461331    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn4d2\" (UniqueName: \"kubernetes.io/projected/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-kube-api-access-gn4d2\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:05.461375 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.461360    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8st8\" (UniqueName: \"kubernetes.io/projected/301f5fdf-8bd3-49b3-be41-5684c210577c-kube-api-access-k8st8\") pod \"ingress-canary-gflvk\" (UID: \"301f5fdf-8bd3-49b3-be41-5684c210577c\") " pod="openshift-ingress-canary/ingress-canary-gflvk"
Nov 04 02:06:05.461375 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.461316    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-certificates\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:06:05.461556 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.461386    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vb2x\" (UniqueName: \"kubernetes.io/projected/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-kube-api-access-8vb2x\") pod \"cluster-monitoring-operator-884576c57-btjpg\" (UID: \"63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg"
Nov 04 02:06:05.461556 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.461411    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2v2n\" (UniqueName: \"kubernetes.io/projected/150189f8-cd39-4a16-9d07-112edc6e83dc-kube-api-access-j2v2n\") pod \"service-ca-operator-544d749c8c-c2vfk\" (UID: \"150189f8-cd39-4a16-9d07-112edc6e83dc\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-c2vfk"
Nov 04 02:06:05.461556 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.461452    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-systemd-units\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.461556 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.461477    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-modprobe-d\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.461556 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.461500    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-sysctl-conf\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.461556 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.461537    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pl9r\" (UniqueName: \"kubernetes.io/projected/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-kube-api-access-9pl9r\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.461804 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.461592    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70aa0113-0423-4bdf-8335-d6c5bd707757-tmp-dir\") pod \"node-resolver-vjp7p\" (UID: \"70aa0113-0423-4bdf-8335-d6c5bd707757\") " pod="openshift-dns/node-resolver-vjp7p"
Nov 04 02:06:05.461804 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.461719    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70aa0113-0423-4bdf-8335-d6c5bd707757-tmp-dir\") pod \"node-resolver-vjp7p\" (UID: \"70aa0113-0423-4bdf-8335-d6c5bd707757\") " pod="openshift-dns/node-resolver-vjp7p"
Nov 04 02:06:05.461804 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.461742    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-cni-binary-copy\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.461804 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.461770    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:06:05.461937 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.461806    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cab3ee15-5d68-4840-83f4-ea8e2308e530-installation-pull-secrets\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:06:05.461937 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.461831    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vg7p\" (UniqueName: \"kubernetes.io/projected/430bb96c-bf05-4a46-bdd3-612bd19a26d8-kube-api-access-6vg7p\") pod \"migrator-66df5d496b-nvl6s\" (UID: \"430bb96c-bf05-4a46-bdd3-612bd19a26d8\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nvl6s"
Nov 04 02:06:05.461937 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.461914    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/301f5fdf-8bd3-49b3-be41-5684c210577c-cert\") pod \"ingress-canary-gflvk\" (UID: \"301f5fdf-8bd3-49b3-be41-5684c210577c\") " pod="openshift-ingress-canary/ingress-canary-gflvk"
Nov 04 02:06:05.462068 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.461999    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3be2e6a4-4e32-4eb9-a4cf-1ce88c6e2237-signing-cabundle\") pod \"service-ca-5fffc97bdf-pwwpf\" (UID: \"3be2e6a4-4e32-4eb9-a4cf-1ce88c6e2237\") " pod="openshift-service-ca/service-ca-5fffc97bdf-pwwpf"
Nov 04 02:06:05.462068 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462036    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cab3ee15-5d68-4840-83f4-ea8e2308e530-installation-pull-secrets\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:06:05.462068 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462044    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-cni-bin\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.462228 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462082    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e164905d-5505-4d20-af2e-b8acd370bf18-dbus\") pod \"global-pull-secret-syncer-b4wjr\" (UID: \"e164905d-5505-4d20-af2e-b8acd370bf18\") " pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:06:05.462228 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462133    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9a395e26-3eac-4728-bbad-c42eaf6da56d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:06:05.462228 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462150    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:06:05.462228 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462166    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7cdf1fc0-00ea-4108-8ff0-5c031a193968-snapshots\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:06:05.462228 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462204    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-run-k8s-cni-cncf-io\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.462228 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462223    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:06:05.462498 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462243    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-node-log\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.462498 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462286    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e164905d-5505-4d20-af2e-b8acd370bf18-kubelet-config\") pod \"global-pull-secret-syncer-b4wjr\" (UID: \"e164905d-5505-4d20-af2e-b8acd370bf18\") " pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:06:05.462498 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462314    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvghh\" (UniqueName: \"kubernetes.io/projected/63254ae3-0393-4094-9401-e18067d30080-kube-api-access-rvghh\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:06:05.462498 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462341    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-default-certificate\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:06:05.462498 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462364    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnpf6\" (UniqueName: \"kubernetes.io/projected/7cdf1fc0-00ea-4108-8ff0-5c031a193968-kube-api-access-cnpf6\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:06:05.462498 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462380    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-sysconfig\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.462498 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462395    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-multus-cni-dir\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.462498 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462411    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7cdf1fc0-00ea-4108-8ff0-5c031a193968-snapshots\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:06:05.462498 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462431    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-slash\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.462498 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462477    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm6tq\" (UniqueName: \"kubernetes.io/projected/eb9e13cc-14be-46c7-bea5-2501b0a85486-kube-api-access-qm6tq\") pod \"periodic-gathering-xwzqv-7rqm2\" (UID: \"eb9e13cc-14be-46c7-bea5-2501b0a85486\") " pod="openshift-insights/periodic-gathering-xwzqv-7rqm2"
Nov 04 02:06:05.462930 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462504    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/16f00163-6ec3-4c10-81e4-c8a501e15e82-config\") pod \"console-operator-7cd56b8dbc-jcqmp\" (UID: \"16f00163-6ec3-4c10-81e4-c8a501e15e82\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"
Nov 04 02:06:05.462930 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462529    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-etc-kubernetes\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.462930 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462555    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkch2\" (UniqueName: \"kubernetes.io/projected/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-kube-api-access-hkch2\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.462930 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462626    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e164905d-5505-4d20-af2e-b8acd370bf18-original-pull-secret\") pod \"global-pull-secret-syncer-b4wjr\" (UID: \"e164905d-5505-4d20-af2e-b8acd370bf18\") " pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:06:05.462930 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462659    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb9e13cc-14be-46c7-bea5-2501b0a85486-service-ca-bundle\") pod \"periodic-gathering-xwzqv-7rqm2\" (UID: \"eb9e13cc-14be-46c7-bea5-2501b0a85486\") " pod="openshift-insights/periodic-gathering-xwzqv-7rqm2"
Nov 04 02:06:05.462930 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462688    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a395e26-3eac-4728-bbad-c42eaf6da56d-cni-binary-copy\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:06:05.462930 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462714    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-kubernetes\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.462930 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462740    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-tuned\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.462930 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462767    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/150189f8-cd39-4a16-9d07-112edc6e83dc-serving-cert\") pod \"service-ca-operator-544d749c8c-c2vfk\" (UID: \"150189f8-cd39-4a16-9d07-112edc6e83dc\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-c2vfk"
Nov 04 02:06:05.462930 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462781    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-default-certificate\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:06:05.462930 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462793    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-cni-netd\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.462930 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462826    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-tuned\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.462930 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462839    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-ovn-node-metrics-cert\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.462930 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462828    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb9e13cc-14be-46c7-bea5-2501b0a85486-service-ca-bundle\") pod \"periodic-gathering-xwzqv-7rqm2\" (UID: \"eb9e13cc-14be-46c7-bea5-2501b0a85486\") " pod="openshift-insights/periodic-gathering-xwzqv-7rqm2"
Nov 04 02:06:05.462930 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462867    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqvtx\" (UniqueName: \"kubernetes.io/projected/1597f8a8-5b13-4076-9d95-c494cf3ad5dc-kube-api-access-nqvtx\") pod \"kas-connection-checker-685895c7c9-d27bw\" (UID: \"1597f8a8-5b13-4076-9d95-c494cf3ad5dc\") " pod="kube-system/kas-connection-checker-685895c7c9-d27bw"
Nov 04 02:06:05.462930 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462902    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-h72td\" (UID: \"3f0365d3-076c-4128-9349-b7dd71181e5f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"
Nov 04 02:06:05.463492 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.462998    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:06:05.463492 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.463019    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/150189f8-cd39-4a16-9d07-112edc6e83dc-config\") pod \"service-ca-operator-544d749c8c-c2vfk\" (UID: \"150189f8-cd39-4a16-9d07-112edc6e83dc\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-c2vfk"
Nov 04 02:06:05.463492 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.463035    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-96g6r\" (UID: \"a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-96g6r"
Nov 04 02:06:05.463492 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.463053    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-96g6r\" (UID: \"a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-96g6r"
Nov 04 02:06:05.463492 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.463084    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-run-netns\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.463492 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.463122    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-h72td\" (UID: \"3f0365d3-076c-4128-9349-b7dd71181e5f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"
Nov 04 02:06:05.463492 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.463251    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-96g6r\" (UID: \"a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-96g6r"
Nov 04 02:06:05.463492 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.463316    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:05.463492 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.463344    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-host\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.463492 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.463344    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-96g6r\" (UID: \"a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-96g6r"
Nov 04 02:06:05.463492 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.463377    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-btjpg\" (UID: \"63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg"
Nov 04 02:06:05.463492 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.463421    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8ddd2cf3-5cce-41a8-ad26-6946d3cb5910-serviceca\") pod \"node-ca-gzg65\" (UID: \"8ddd2cf3-5cce-41a8-ad26-6946d3cb5910\") " pod="openshift-image-registry/node-ca-gzg65"
Nov 04 02:06:05.463492 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.463444    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-image-registry-private-configuration\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:06:05.463492 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.463461    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-run-openvswitch\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.464043 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.463653    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-image-registry-private-configuration\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:06:05.464043 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.463667    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-log-socket\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.464043 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.463687    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6l5g\" (UniqueName: \"kubernetes.io/projected/abdfaf94-b721-4ed4-8671-416e694df5bb-kube-api-access-l6l5g\") pod \"kas-connection-checker-685895c7c9-2mklg\" (UID: \"abdfaf94-b721-4ed4-8671-416e694df5bb\") " pod="kube-system/kas-connection-checker-685895c7c9-2mklg"
Nov 04 02:06:05.464043 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.463706    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/16f00163-6ec3-4c10-81e4-c8a501e15e82-serving-cert\") pod \"console-operator-7cd56b8dbc-jcqmp\" (UID: \"16f00163-6ec3-4c10-81e4-c8a501e15e82\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"
Nov 04 02:06:05.464043 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.463722    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4gp2\" (UniqueName: \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-kube-api-access-l4gp2\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:06:05.464043 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.463738    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-sysctl-d\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.464043 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.463760    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9a395e26-3eac-4728-bbad-c42eaf6da56d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:06:05.464043 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.463803    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8ddd2cf3-5cce-41a8-ad26-6946d3cb5910-serviceca\") pod \"node-ca-gzg65\" (UID: \"8ddd2cf3-5cce-41a8-ad26-6946d3cb5910\") " pod="openshift-image-registry/node-ca-gzg65"
Nov 04 02:06:05.464043 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.463812    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-kubelet\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.464043 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.463837    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l89wj\" (UniqueName: \"kubernetes.io/projected/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-kube-api-access-l89wj\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.464043 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.463875    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/39235bb9-36b4-43e4-9161-043a3fb6a261-konnectivity-ca\") pod \"konnectivity-agent-928nm\" (UID: \"39235bb9-36b4-43e4-9161-043a3fb6a261\") " pod="kube-system/konnectivity-agent-928nm"
Nov 04 02:06:05.464043 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.463900    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-registration-dir\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:06:05.464043 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.463936    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7cdf1fc0-00ea-4108-8ff0-5c031a193968-serving-cert\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:06:05.464043 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.463973    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a395e26-3eac-4728-bbad-c42eaf6da56d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:06:05.464043 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.463991    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85d9k\" (UniqueName: \"kubernetes.io/projected/16c89b3c-f63f-4489-8eae-5918677e1e61-kube-api-access-85d9k\") pod \"iptables-alerter-qvk5m\" (UID: \"16c89b3c-f63f-4489-8eae-5918677e1e61\") " pod="openshift-network-operator/iptables-alerter-qvk5m"
Nov 04 02:06:05.464043 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464007    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-stats-auth\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:06:05.464043 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464032    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-multus-daemon-config\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.464711 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464056    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-trusted-ca\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:06:05.464711 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464072    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-env-overrides\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.464711 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464086    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7cdf1fc0-00ea-4108-8ff0-5c031a193968-serving-cert\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:06:05.464711 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464102    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/eb9e13cc-14be-46c7-bea5-2501b0a85486-archives-path\") pod \"periodic-gathering-xwzqv-7rqm2\" (UID: \"eb9e13cc-14be-46c7-bea5-2501b0a85486\") " pod="openshift-insights/periodic-gathering-xwzqv-7rqm2"
Nov 04 02:06:05.464711 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464141    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a395e26-3eac-4728-bbad-c42eaf6da56d-system-cni-dir\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:06:05.464711 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464165    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/eb9e13cc-14be-46c7-bea5-2501b0a85486-archives-path\") pod \"periodic-gathering-xwzqv-7rqm2\" (UID: \"eb9e13cc-14be-46c7-bea5-2501b0a85486\") " pod="openshift-insights/periodic-gathering-xwzqv-7rqm2"
Nov 04 02:06:05.464711 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464172    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-btjpg\" (UID: \"63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg"
Nov 04 02:06:05.464711 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464205    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/63254ae3-0393-4094-9401-e18067d30080-config-volume\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:06:05.464711 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464227    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-var-lib-kubelet\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.464711 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464249    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cab3ee15-5d68-4840-83f4-ea8e2308e530-ca-trust-extracted\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:06:05.464711 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464228    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-stats-auth\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:06:05.464711 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464333    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cab3ee15-5d68-4840-83f4-ea8e2308e530-ca-trust-extracted\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:06:05.464711 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464375    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcv2t\" (UniqueName: \"kubernetes.io/projected/70aa0113-0423-4bdf-8335-d6c5bd707757-kube-api-access-fcv2t\") pod \"node-resolver-vjp7p\" (UID: \"70aa0113-0423-4bdf-8335-d6c5bd707757\") " pod="openshift-dns/node-resolver-vjp7p"
Nov 04 02:06:05.464711 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464411    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cab3ee15-5d68-4840-83f4-ea8e2308e530-image-registry-private-configuration\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:06:05.464711 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464383    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-btjpg\" (UID: \"63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg"
Nov 04 02:06:05.464711 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464442    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x24q6\" (UniqueName: \"kubernetes.io/projected/a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d-kube-api-access-x24q6\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-96g6r\" (UID: \"a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-96g6r"
Nov 04 02:06:05.464711 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464440    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-trusted-ca\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:06:05.464711 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464474    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-device-dir\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:06:05.464711 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464501    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-systemd\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.464711 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464526    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/16c89b3c-f63f-4489-8eae-5918677e1e61-host-slash\") pod \"iptables-alerter-qvk5m\" (UID: \"16c89b3c-f63f-4489-8eae-5918677e1e61\") " pod="openshift-network-operator/iptables-alerter-qvk5m"
Nov 04 02:06:05.464711 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464553    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr84q\" (UniqueName: \"kubernetes.io/projected/afe4b3d8-a18c-4587-9057-83123ffe3347-kube-api-access-fr84q\") pod \"network-metrics-daemon-4v7xm\" (UID: \"afe4b3d8-a18c-4587-9057-83123ffe3347\") " pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:06:05.464711 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464562    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cab3ee15-5d68-4840-83f4-ea8e2308e530-image-registry-private-configuration\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:06:05.464711 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464581    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-run-ovn-kubernetes\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.464711 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464617    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.464711 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464676    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:06:05.464711 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464704    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-multus-conf-dir\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.464711 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464723    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/39235bb9-36b4-43e4-9161-043a3fb6a261-agent-certs\") pod \"konnectivity-agent-928nm\" (UID: \"39235bb9-36b4-43e4-9161-043a3fb6a261\") " pod="kube-system/konnectivity-agent-928nm"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464774    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9j8z\" (UniqueName: \"kubernetes.io/projected/7c365e54-8c98-4271-a66e-9b13fe91afa7-kube-api-access-s9j8z\") pod \"migrator-66df5d496b-dcf2j\" (UID: \"7c365e54-8c98-4271-a66e-9b13fe91afa7\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-dcf2j"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464806    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cab3ee15-5d68-4840-83f4-ea8e2308e530-trusted-ca\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464839    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-bound-sa-token\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464866    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cdf1fc0-00ea-4108-8ff0-5c031a193968-service-ca-bundle\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464898    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrwzs\" (UniqueName: \"kubernetes.io/projected/37d0fd53-4873-4d8e-9472-6754d577c9a5-kube-api-access-mrwzs\") pod \"network-check-source-6f78d59f99-62blg\" (UID: \"37d0fd53-4873-4d8e-9472-6754d577c9a5\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-62blg"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464922    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63254ae3-0393-4094-9401-e18067d30080-tmp-dir\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.464948    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcfqp\" (UniqueName: \"kubernetes.io/projected/32699b90-bd7d-43f5-bdcc-53de9da6f244-kube-api-access-qcfqp\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465034    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63254ae3-0393-4094-9401-e18067d30080-tmp-dir\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465109    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-run-systemd\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465141    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465142    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-run-netns\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465255    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465260    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465260    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-var-lib-cni-multus\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465291    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465316    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-bound-sa-token\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465344    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cdf1fc0-00ea-4108-8ff0-5c031a193968-service-ca-bundle\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465353    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-installation-pull-secrets\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465378    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465386    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-etc-selinux\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465419    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-sys-fs\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465448    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7cdf1fc0-00ea-4108-8ff0-5c031a193968-tmp\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465455    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465478    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-var-lib-kubelet\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465510    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-tmp\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465544    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h77nn\" (UniqueName: \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-kube-api-access-h77nn\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465550    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465596    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-installation-pull-secrets\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465603    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465643    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-tmp\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465646    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465674    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-etc-openvswitch\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465675    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7cdf1fc0-00ea-4108-8ff0-5c031a193968-tmp\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465715    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-ovnkube-config\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465733    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465740    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cab3ee15-5d68-4840-83f4-ea8e2308e530-trusted-ca\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465760    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-crio-socket\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465647    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465782    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465791    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7f7m\" (UniqueName: \"kubernetes.io/projected/b2156825-c0d8-48a4-a90f-2340e98a13e3-kube-api-access-b7f7m\") pod \"kas-connection-checker-685895c7c9-pn5gm\" (UID: \"b2156825-c0d8-48a4-a90f-2340e98a13e3\") " pod="kube-system/kas-connection-checker-685895c7c9-pn5gm"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465811    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465822    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465826    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465849    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465867    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465881    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465882    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-socket-dir\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465898    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 04 02:06:05.465880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465916    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465928    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465945    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465948    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465971    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465977    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465998    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.466013    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.466015    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.466015    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.466039    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16f00163-6ec3-4c10-81e4-c8a501e15e82-trusted-ca\") pod \"console-operator-7cd56b8dbc-jcqmp\" (UID: \"16f00163-6ec3-4c10-81e4-c8a501e15e82\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.466047    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.466068    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.465946    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.466077    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.466085    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ddd2cf3-5cce-41a8-ad26-6946d3cb5910-host\") pod \"node-ca-gzg65\" (UID: \"8ddd2cf3-5cce-41a8-ad26-6946d3cb5910\") " pod="openshift-image-registry/node-ca-gzg65"
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.466109    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.466134    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.466162    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.466183    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcxzt\" (UniqueName: \"kubernetes.io/projected/8ddd2cf3-5cce-41a8-ad26-6946d3cb5910-kube-api-access-lcxzt\") pod \"node-ca-gzg65\" (UID: \"8ddd2cf3-5cce-41a8-ad26-6946d3cb5910\") " pod="openshift-image-registry/node-ca-gzg65"
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.466303    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3be2e6a4-4e32-4eb9-a4cf-1ce88c6e2237-signing-key\") pod \"service-ca-5fffc97bdf-pwwpf\" (UID: \"3be2e6a4-4e32-4eb9-a4cf-1ce88c6e2237\") " pod="openshift-service-ca/service-ca-5fffc97bdf-pwwpf"
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.466406    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.466407    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-certificates\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.466455    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-run\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.466465    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.466492    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-lib-modules\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.466529    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/70aa0113-0423-4bdf-8335-d6c5bd707757-hosts-file\") pod \"node-resolver-vjp7p\" (UID: \"70aa0113-0423-4bdf-8335-d6c5bd707757\") " pod="openshift-dns/node-resolver-vjp7p"
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.466536    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-certificates\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.466591    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.466595    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-cnibin\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.466644    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-multus-socket-dir-parent\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.466675    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q8rb\" (UniqueName: \"kubernetes.io/projected/3be2e6a4-4e32-4eb9-a4cf-1ce88c6e2237-kube-api-access-2q8rb\") pod \"service-ca-5fffc97bdf-pwwpf\" (UID: \"3be2e6a4-4e32-4eb9-a4cf-1ce88c6e2237\") " pod="openshift-service-ca/service-ca-5fffc97bdf-pwwpf"
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.467063    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-nginx-conf\") pod \"networking-console-plugin-86c478bb88-pxvln\" (UID: \"0851f8eb-225e-4576-b5bc-dc6d72eefb6c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pxvln"
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.467134    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/16c89b3c-f63f-4489-8eae-5918677e1e61-iptables-alerter-script\") pod \"iptables-alerter-qvk5m\" (UID: \"16c89b3c-f63f-4489-8eae-5918677e1e61\") " pod="openshift-network-operator/iptables-alerter-qvk5m"
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.467177    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96kfp\" (UniqueName: \"kubernetes.io/projected/16f00163-6ec3-4c10-81e4-c8a501e15e82-kube-api-access-96kfp\") pod \"console-operator-7cd56b8dbc-jcqmp\" (UID: \"16f00163-6ec3-4c10-81e4-c8a501e15e82\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.467202    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-nginx-conf\") pod \"networking-console-plugin-86c478bb88-pxvln\" (UID: \"0851f8eb-225e-4576-b5bc-dc6d72eefb6c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pxvln"
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.467213    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-system-cni-dir\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.467248    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-482t2\" (UniqueName: \"kubernetes.io/projected/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-kube-api-access-482t2\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.467510    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32699b90-bd7d-43f5-bdcc-53de9da6f244-service-ca-bundle\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.467525    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/16c89b3c-f63f-4489-8eae-5918677e1e61-iptables-alerter-script\") pod \"iptables-alerter-qvk5m\" (UID: \"16c89b3c-f63f-4489-8eae-5918677e1e61\") " pod="openshift-network-operator/iptables-alerter-qvk5m"
Nov 04 02:06:05.468025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.467595    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84hdg\" (UniqueName: \"kubernetes.io/projected/9a395e26-3eac-4728-bbad-c42eaf6da56d-kube-api-access-84hdg\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:06:05.470058 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.467630    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pxvln\" (UID: \"0851f8eb-225e-4576-b5bc-dc6d72eefb6c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pxvln"
Nov 04 02:06:05.470058 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.468148    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-run-ovn\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.470058 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.468183    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-sys\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.470058 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.468200    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Nov 04 02:06:05.470058 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.468225    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjm7q\" (UniqueName: \"kubernetes.io/projected/8ec1a73f-2254-499d-ab62-96c8c223ed8d-kube-api-access-kjm7q\") pod \"volume-data-source-validator-58c9f5f46d-xxtnw\" (UID: \"8ec1a73f-2254-499d-ab62-96c8c223ed8d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xxtnw"
Nov 04 02:06:05.470058 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.468293    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a395e26-3eac-4728-bbad-c42eaf6da56d-cnibin\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:06:05.470058 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.468339    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-run-multus-certs\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.470058 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.468396    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-ca-trust-extracted\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:06:05.470058 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.468436    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:05.470058 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.468514    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cdf1fc0-00ea-4108-8ff0-5c031a193968-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:06:05.470058 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.468553    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrkqj\" (UniqueName: \"kubernetes.io/projected/b0a72217-fe1d-4589-9201-651503694cb4-kube-api-access-nrkqj\") pod \"network-check-target-98h6z\" (UID: \"b0a72217-fe1d-4589-9201-651503694cb4\") " pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:06:05.470058 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.468589    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-hostroot\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.470058 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.468846    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:05.470058 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.468915    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-ca-trust-extracted\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:06:05.470058 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.469163    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cdf1fc0-00ea-4108-8ff0-5c031a193968-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472111    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-ovnkube-script-lib\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472185    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472237    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-cni-binary-copy\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472266    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472344    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472419    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472429    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472445    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9a395e26-3eac-4728-bbad-c42eaf6da56d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472453    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-fm245\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472460    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-4gr4n\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472491    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472524    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472532    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472552    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472557    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472553    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-75dtl\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472609    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-8fdm8\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472629    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-db624\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472660    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-h6pb6\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472638    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pzhlz\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472675    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-szjlx\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472710    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-j7wvp\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472715    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ckptg\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472728    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qq5vw\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472747    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-kn9lr\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472765    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-h9jgb\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472777    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z899q\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472780    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kvkfc\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472789    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-mbl7d\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472791    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/16f00163-6ec3-4c10-81e4-c8a501e15e82-config\") pod \"console-operator-7cd56b8dbc-jcqmp\" (UID: \"16f00163-6ec3-4c10-81e4-c8a501e15e82\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472823    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-4f6g8\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472850    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472906    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a395e26-3eac-4728-bbad-c42eaf6da56d-cni-binary-copy\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472919    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-mgqv4\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472950    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/150189f8-cd39-4a16-9d07-112edc6e83dc-serving-cert\") pod \"service-ca-operator-544d749c8c-c2vfk\" (UID: \"150189f8-cd39-4a16-9d07-112edc6e83dc\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-c2vfk"
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.472978    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-ghp2x\""
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.473023    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-ovn-node-metrics-cert\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.473051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.473035    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 02:06:05.474554 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.473087    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rtznt\""
Nov 04 02:06:05.474554 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.473303    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-zd6qr\""
Nov 04 02:06:05.474554 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.473381    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/150189f8-cd39-4a16-9d07-112edc6e83dc-config\") pod \"service-ca-operator-544d749c8c-c2vfk\" (UID: \"150189f8-cd39-4a16-9d07-112edc6e83dc\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-c2vfk"
Nov 04 02:06:05.474554 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.473982    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/16f00163-6ec3-4c10-81e4-c8a501e15e82-serving-cert\") pod \"console-operator-7cd56b8dbc-jcqmp\" (UID: \"16f00163-6ec3-4c10-81e4-c8a501e15e82\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"
Nov 04 02:06:05.474554 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.474168    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/39235bb9-36b4-43e4-9161-043a3fb6a261-konnectivity-ca\") pod \"konnectivity-agent-928nm\" (UID: \"39235bb9-36b4-43e4-9161-043a3fb6a261\") " pod="kube-system/konnectivity-agent-928nm"
Nov 04 02:06:05.474554 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.474172    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9a395e26-3eac-4728-bbad-c42eaf6da56d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:06:05.474554 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.474264    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-env-overrides\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.474554 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.474392    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-multus-daemon-config\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.474554 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.474421    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/63254ae3-0393-4094-9401-e18067d30080-config-volume\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:06:05.475224 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.475210    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/39235bb9-36b4-43e4-9161-043a3fb6a261-agent-certs\") pod \"konnectivity-agent-928nm\" (UID: \"39235bb9-36b4-43e4-9161-043a3fb6a261\") " pod="kube-system/konnectivity-agent-928nm"
Nov 04 02:06:05.476312 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.476296    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-ovnkube-config\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.476408 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.476395    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/16f00163-6ec3-4c10-81e4-c8a501e15e82-trusted-ca\") pod \"console-operator-7cd56b8dbc-jcqmp\" (UID: \"16f00163-6ec3-4c10-81e4-c8a501e15e82\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"
Nov 04 02:06:05.482576 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.482561    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/301f5fdf-8bd3-49b3-be41-5684c210577c-cert\") pod \"ingress-canary-gflvk\" (UID: \"301f5fdf-8bd3-49b3-be41-5684c210577c\") " pod="openshift-ingress-canary/ingress-canary-gflvk"
Nov 04 02:06:05.483753 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.483738    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e164905d-5505-4d20-af2e-b8acd370bf18-original-pull-secret\") pod \"global-pull-secret-syncer-b4wjr\" (UID: \"e164905d-5505-4d20-af2e-b8acd370bf18\") " pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:06:05.523875 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.523852    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2v2n\" (UniqueName: \"kubernetes.io/projected/150189f8-cd39-4a16-9d07-112edc6e83dc-kube-api-access-j2v2n\") pod \"service-ca-operator-544d749c8c-c2vfk\" (UID: \"150189f8-cd39-4a16-9d07-112edc6e83dc\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-c2vfk"
Nov 04 02:06:05.528894 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.528872    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vg7p\" (UniqueName: \"kubernetes.io/projected/430bb96c-bf05-4a46-bdd3-612bd19a26d8-kube-api-access-6vg7p\") pod \"migrator-66df5d496b-nvl6s\" (UID: \"430bb96c-bf05-4a46-bdd3-612bd19a26d8\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nvl6s"
Nov 04 02:06:05.528960 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.528913    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pl9r\" (UniqueName: \"kubernetes.io/projected/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-kube-api-access-9pl9r\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.529073 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.529058    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vb2x\" (UniqueName: \"kubernetes.io/projected/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-kube-api-access-8vb2x\") pod \"cluster-monitoring-operator-884576c57-btjpg\" (UID: \"63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg"
Nov 04 02:06:05.529739 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.529719    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn4d2\" (UniqueName: \"kubernetes.io/projected/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-kube-api-access-gn4d2\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:05.529932 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.529915    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8st8\" (UniqueName: \"kubernetes.io/projected/301f5fdf-8bd3-49b3-be41-5684c210577c-kube-api-access-k8st8\") pod \"ingress-canary-gflvk\" (UID: \"301f5fdf-8bd3-49b3-be41-5684c210577c\") " pod="openshift-ingress-canary/ingress-canary-gflvk"
Nov 04 02:06:05.529964 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.529946    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnpf6\" (UniqueName: \"kubernetes.io/projected/7cdf1fc0-00ea-4108-8ff0-5c031a193968-kube-api-access-cnpf6\") pod \"insights-operator-7c4986d449-6cc2t\" (UID: \"7cdf1fc0-00ea-4108-8ff0-5c031a193968\") " pod="openshift-insights/insights-operator-7c4986d449-6cc2t"
Nov 04 02:06:05.530247 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.530232    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvghh\" (UniqueName: \"kubernetes.io/projected/63254ae3-0393-4094-9401-e18067d30080-kube-api-access-rvghh\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:06:05.535638 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.535621    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98v7l\" (UniqueName: \"kubernetes.io/projected/3f0365d3-076c-4128-9349-b7dd71181e5f-kube-api-access-98v7l\") pod \"cluster-samples-operator-6ddb6c8b94-h72td\" (UID: \"3f0365d3-076c-4128-9349-b7dd71181e5f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"
Nov 04 02:06:05.539219 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.539203    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4gp2\" (UniqueName: \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-kube-api-access-l4gp2\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:06:05.539387 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.539371    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrwzs\" (UniqueName: \"kubernetes.io/projected/37d0fd53-4873-4d8e-9472-6754d577c9a5-kube-api-access-mrwzs\") pod \"network-check-source-6f78d59f99-62blg\" (UID: \"37d0fd53-4873-4d8e-9472-6754d577c9a5\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-62blg"
Nov 04 02:06:05.539990 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.539963    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l89wj\" (UniqueName: \"kubernetes.io/projected/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-kube-api-access-l89wj\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.541655 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.541627    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-bound-sa-token\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:06:05.542431 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.542413    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqvtx\" (UniqueName: \"kubernetes.io/projected/1597f8a8-5b13-4076-9d95-c494cf3ad5dc-kube-api-access-nqvtx\") pod \"kas-connection-checker-685895c7c9-d27bw\" (UID: \"1597f8a8-5b13-4076-9d95-c494cf3ad5dc\") " pod="kube-system/kas-connection-checker-685895c7c9-d27bw"
Nov 04 02:06:05.542482 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.542464    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6l5g\" (UniqueName: \"kubernetes.io/projected/abdfaf94-b721-4ed4-8671-416e694df5bb-kube-api-access-l6l5g\") pod \"kas-connection-checker-685895c7c9-2mklg\" (UID: \"abdfaf94-b721-4ed4-8671-416e694df5bb\") " pod="kube-system/kas-connection-checker-685895c7c9-2mklg"
Nov 04 02:06:05.542591 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.542574    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcv2t\" (UniqueName: \"kubernetes.io/projected/70aa0113-0423-4bdf-8335-d6c5bd707757-kube-api-access-fcv2t\") pod \"node-resolver-vjp7p\" (UID: \"70aa0113-0423-4bdf-8335-d6c5bd707757\") " pod="openshift-dns/node-resolver-vjp7p"
Nov 04 02:06:05.542935 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.542919    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-482t2\" (UniqueName: \"kubernetes.io/projected/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-kube-api-access-482t2\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:06:05.547229 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.547210    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84hdg\" (UniqueName: \"kubernetes.io/projected/9a395e26-3eac-4728-bbad-c42eaf6da56d-kube-api-access-84hdg\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:06:05.547350 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.547330    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-bound-sa-token\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:06:05.547422 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.547409    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcxzt\" (UniqueName: \"kubernetes.io/projected/8ddd2cf3-5cce-41a8-ad26-6946d3cb5910-kube-api-access-lcxzt\") pod \"node-ca-gzg65\" (UID: \"8ddd2cf3-5cce-41a8-ad26-6946d3cb5910\") " pod="openshift-image-registry/node-ca-gzg65"
Nov 04 02:06:05.547683 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.547668    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr84q\" (UniqueName: \"kubernetes.io/projected/afe4b3d8-a18c-4587-9057-83123ffe3347-kube-api-access-fr84q\") pod \"network-metrics-daemon-4v7xm\" (UID: \"afe4b3d8-a18c-4587-9057-83123ffe3347\") " pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:06:05.547946 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.547927    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrkqj\" (UniqueName: \"kubernetes.io/projected/b0a72217-fe1d-4589-9201-651503694cb4-kube-api-access-nrkqj\") pod \"network-check-target-98h6z\" (UID: \"b0a72217-fe1d-4589-9201-651503694cb4\") " pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:06:05.548148 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.548123    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x24q6\" (UniqueName: \"kubernetes.io/projected/a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d-kube-api-access-x24q6\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-96g6r\" (UID: \"a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-96g6r"
Nov 04 02:06:05.548264 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.548248    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjm7q\" (UniqueName: \"kubernetes.io/projected/8ec1a73f-2254-499d-ab62-96c8c223ed8d-kube-api-access-kjm7q\") pod \"volume-data-source-validator-58c9f5f46d-xxtnw\" (UID: \"8ec1a73f-2254-499d-ab62-96c8c223ed8d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-xxtnw"
Nov 04 02:06:05.548352 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.548328    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h77nn\" (UniqueName: \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-kube-api-access-h77nn\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:06:05.552979 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.552960    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85d9k\" (UniqueName: \"kubernetes.io/projected/16c89b3c-f63f-4489-8eae-5918677e1e61-kube-api-access-85d9k\") pod \"iptables-alerter-qvk5m\" (UID: \"16c89b3c-f63f-4489-8eae-5918677e1e61\") " pod="openshift-network-operator/iptables-alerter-qvk5m"
Nov 04 02:06:05.553110 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.553064    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkch2\" (UniqueName: \"kubernetes.io/projected/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-kube-api-access-hkch2\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.553158 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.553069    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7f7m\" (UniqueName: \"kubernetes.io/projected/b2156825-c0d8-48a4-a90f-2340e98a13e3-kube-api-access-b7f7m\") pod \"kas-connection-checker-685895c7c9-pn5gm\" (UID: \"b2156825-c0d8-48a4-a90f-2340e98a13e3\") " pod="kube-system/kas-connection-checker-685895c7c9-pn5gm"
Nov 04 02:06:05.553386 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.553367    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm6tq\" (UniqueName: \"kubernetes.io/projected/eb9e13cc-14be-46c7-bea5-2501b0a85486-kube-api-access-qm6tq\") pod \"periodic-gathering-xwzqv-7rqm2\" (UID: \"eb9e13cc-14be-46c7-bea5-2501b0a85486\") " pod="openshift-insights/periodic-gathering-xwzqv-7rqm2"
Nov 04 02:06:05.553443 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.553429    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9j8z\" (UniqueName: \"kubernetes.io/projected/7c365e54-8c98-4271-a66e-9b13fe91afa7-kube-api-access-s9j8z\") pod \"migrator-66df5d496b-dcf2j\" (UID: \"7c365e54-8c98-4271-a66e-9b13fe91afa7\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-dcf2j"
Nov 04 02:06:05.553665 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.553650    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcfqp\" (UniqueName: \"kubernetes.io/projected/32699b90-bd7d-43f5-bdcc-53de9da6f244-kube-api-access-qcfqp\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:06:05.554929 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.554913    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96kfp\" (UniqueName: \"kubernetes.io/projected/16f00163-6ec3-4c10-81e4-c8a501e15e82-kube-api-access-96kfp\") pod \"console-operator-7cd56b8dbc-jcqmp\" (UID: \"16f00163-6ec3-4c10-81e4-c8a501e15e82\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"
Nov 04 02:06:05.569779 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.569758    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-multus-conf-dir\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.569841 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.569787    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-run-systemd\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.569841 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.569803    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-run-netns\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.569841 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.569817    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-var-lib-cni-multus\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.569841 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.569833    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-etc-selinux\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:06:05.570005 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.569847    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-sys-fs\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:06:05.570005 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.569861    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-var-lib-kubelet\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.570005 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.569862    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-run-systemd\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.570005 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.569867    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-run-netns\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.570005 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.569883    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-etc-openvswitch\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.570005 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.569908    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-var-lib-cni-multus\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.570005 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.569919    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-etc-openvswitch\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.570005 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.569917    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-crio-socket\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:05.570005 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.569925    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-sys-fs\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:06:05.570005 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.569923    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-etc-selinux\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:06:05.570005 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.569919    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-var-lib-kubelet\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.570005 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.569952    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-multus-conf-dir\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.570005 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.569953    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-socket-dir\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:06:05.570005 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.569982    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ddd2cf3-5cce-41a8-ad26-6946d3cb5910-host\") pod \"node-ca-gzg65\" (UID: \"8ddd2cf3-5cce-41a8-ad26-6946d3cb5910\") " pod="openshift-image-registry/node-ca-gzg65"
Nov 04 02:06:05.570005 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.569988    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-crio-socket\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570016    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3be2e6a4-4e32-4eb9-a4cf-1ce88c6e2237-signing-key\") pod \"service-ca-5fffc97bdf-pwwpf\" (UID: \"3be2e6a4-4e32-4eb9-a4cf-1ce88c6e2237\") " pod="openshift-service-ca/service-ca-5fffc97bdf-pwwpf"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570033    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-socket-dir\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570048    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ddd2cf3-5cce-41a8-ad26-6946d3cb5910-host\") pod \"node-ca-gzg65\" (UID: \"8ddd2cf3-5cce-41a8-ad26-6946d3cb5910\") " pod="openshift-image-registry/node-ca-gzg65"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570108    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-run\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570135    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-run\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570158    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-lib-modules\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570179    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/70aa0113-0423-4bdf-8335-d6c5bd707757-hosts-file\") pod \"node-resolver-vjp7p\" (UID: \"70aa0113-0423-4bdf-8335-d6c5bd707757\") " pod="openshift-dns/node-resolver-vjp7p"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570231    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-cnibin\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570238    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/70aa0113-0423-4bdf-8335-d6c5bd707757-hosts-file\") pod \"node-resolver-vjp7p\" (UID: \"70aa0113-0423-4bdf-8335-d6c5bd707757\") " pod="openshift-dns/node-resolver-vjp7p"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570256    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-multus-socket-dir-parent\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570267    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-lib-modules\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570281    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q8rb\" (UniqueName: \"kubernetes.io/projected/3be2e6a4-4e32-4eb9-a4cf-1ce88c6e2237-kube-api-access-2q8rb\") pod \"service-ca-5fffc97bdf-pwwpf\" (UID: \"3be2e6a4-4e32-4eb9-a4cf-1ce88c6e2237\") " pod="openshift-service-ca/service-ca-5fffc97bdf-pwwpf"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570290    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-cnibin\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570308    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-system-cni-dir\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570317    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-multus-socket-dir-parent\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570334    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32699b90-bd7d-43f5-bdcc-53de9da6f244-service-ca-bundle\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570349    4969 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570366    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-system-cni-dir\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570361    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pxvln\" (UID: \"0851f8eb-225e-4576-b5bc-dc6d72eefb6c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pxvln"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570428    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-run-ovn\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:05.570439    4969 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570444    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-sys\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570477    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-sys\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570492    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-run-ovn\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:05.570498    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/32699b90-bd7d-43f5-bdcc-53de9da6f244-service-ca-bundle podName:32699b90-bd7d-43f5-bdcc-53de9da6f244 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:06.070478575 +0000 UTC m=+1.996701201 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/32699b90-bd7d-43f5-bdcc-53de9da6f244-service-ca-bundle") pod "router-default-6595f8bf85-dmj26" (UID: "32699b90-bd7d-43f5-bdcc-53de9da6f244") : configmap references non-existent config key: service-ca.crt
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:05.570541    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert podName:0851f8eb-225e-4576-b5bc-dc6d72eefb6c nodeName:}" failed. No retries permitted until 2026-11-04 02:06:06.070529349 +0000 UTC m=+1.996751960 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-pxvln" (UID: "0851f8eb-225e-4576-b5bc-dc6d72eefb6c") : secret "networking-console-plugin-cert" not found
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570557    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a395e26-3eac-4728-bbad-c42eaf6da56d-cnibin\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570578    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-run-multus-certs\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570596    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-hostroot\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570611    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a395e26-3eac-4728-bbad-c42eaf6da56d-cnibin\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570613    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-var-lib-openvswitch\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570646    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-hostroot\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570654    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-h72td\" (UID: \"3f0365d3-076c-4128-9349-b7dd71181e5f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570658    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-run-multus-certs\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570680    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs\") pod \"network-metrics-daemon-4v7xm\" (UID: \"afe4b3d8-a18c-4587-9057-83123ffe3347\") " pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:06:05.570680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570701    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-var-lib-openvswitch\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:05.570706    4969 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570704    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-var-lib-cni-bin\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570750    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-systemd-units\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:05.570765    4969 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570781    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-systemd-units\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570794    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-var-lib-cni-bin\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:05.570768    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls podName:3f0365d3-076c-4128-9349-b7dd71181e5f nodeName:}" failed. No retries permitted until 2026-11-04 02:06:06.070754799 +0000 UTC m=+1.996977429 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-h72td" (UID: "3f0365d3-076c-4128-9349-b7dd71181e5f") : secret "samples-operator-tls" not found
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:05.570842    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs podName:afe4b3d8-a18c-4587-9057-83123ffe3347 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:06.070828072 +0000 UTC m=+1.997050687 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs") pod "network-metrics-daemon-4v7xm" (UID: "afe4b3d8-a18c-4587-9057-83123ffe3347") : secret "metrics-daemon-secret" not found
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570861    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-modprobe-d\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570885    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-sysctl-conf\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570912    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570940    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3be2e6a4-4e32-4eb9-a4cf-1ce88c6e2237-signing-cabundle\") pod \"service-ca-5fffc97bdf-pwwpf\" (UID: \"3be2e6a4-4e32-4eb9-a4cf-1ce88c6e2237\") " pod="openshift-service-ca/service-ca-5fffc97bdf-pwwpf"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570965    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-cni-bin\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570975    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-modprobe-d\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.570987    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e164905d-5505-4d20-af2e-b8acd370bf18-dbus\") pod \"global-pull-secret-syncer-b4wjr\" (UID: \"e164905d-5505-4d20-af2e-b8acd370bf18\") " pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571013    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-sysctl-conf\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571026    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:05.571045    4969 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571051    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-run-k8s-cni-cncf-io\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571054    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-cni-bin\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:05.571058    4969 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bd9d74c6b-thtzz: secret "image-registry-tls" not found
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571067    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e164905d-5505-4d20-af2e-b8acd370bf18-dbus\") pod \"global-pull-secret-syncer-b4wjr\" (UID: \"e164905d-5505-4d20-af2e-b8acd370bf18\") " pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571075    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:05.571125    4969 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571127    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-node-log\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:05.571133    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls podName:cab3ee15-5d68-4840-83f4-ea8e2308e530 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:06.07112035 +0000 UTC m=+1.997342991 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls") pod "image-registry-5bd9d74c6b-thtzz" (UID: "cab3ee15-5d68-4840-83f4-ea8e2308e530") : secret "image-registry-tls" not found
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571160    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e164905d-5505-4d20-af2e-b8acd370bf18-kubelet-config\") pod \"global-pull-secret-syncer-b4wjr\" (UID: \"e164905d-5505-4d20-af2e-b8acd370bf18\") " pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:05.571166    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs podName:32699b90-bd7d-43f5-bdcc-53de9da6f244 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:06.071155027 +0000 UTC m=+1.997377661 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs") pod "router-default-6595f8bf85-dmj26" (UID: "32699b90-bd7d-43f5-bdcc-53de9da6f244") : secret "router-metrics-certs-default" not found
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571166    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-node-log\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571192    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-sysconfig\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571204    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e164905d-5505-4d20-af2e-b8acd370bf18-kubelet-config\") pod \"global-pull-secret-syncer-b4wjr\" (UID: \"e164905d-5505-4d20-af2e-b8acd370bf18\") " pod="kube-system/global-pull-secret-syncer-b4wjr"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571217    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-multus-cni-dir\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:05.571239    4969 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571239    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-slash\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:05.571265    4969 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bc84fd8c9-wbbgn: secret "image-registry-tls" not found
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571269    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-etc-kubernetes\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571288    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-run-k8s-cni-cncf-io\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:05.571298    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls podName:12e9dc19-8f06-4b5d-8237-3ed5a449e09f nodeName:}" failed. No retries permitted until 2026-11-04 02:06:06.071288031 +0000 UTC m=+1.997510673 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls") pod "image-registry-6bc84fd8c9-wbbgn" (UID: "12e9dc19-8f06-4b5d-8237-3ed5a449e09f") : secret "image-registry-tls" not found
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571262    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-sysconfig\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571294    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-multus-cni-dir\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571319    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-etc-kubernetes\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571329    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-slash\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571317    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-kubernetes\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571349    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-kubernetes\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571366    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-cni-netd\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571392    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571409    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-cni-netd\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571416    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-run-netns\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571442    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:05.571459    4969 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571464    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-host\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:05.571488    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls podName:63254ae3-0393-4094-9401-e18067d30080 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:06.07147562 +0000 UTC m=+1.997698231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls") pod "dns-default-8sr5s" (UID: "63254ae3-0393-4094-9401-e18067d30080") : secret "dns-default-metrics-tls" not found
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571498    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-run-netns\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571527    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-host\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571528    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-btjpg\" (UID: \"63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:05.571545    4969 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571562    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-run-openvswitch\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571567    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3be2e6a4-4e32-4eb9-a4cf-1ce88c6e2237-signing-cabundle\") pod \"service-ca-5fffc97bdf-pwwpf\" (UID: \"3be2e6a4-4e32-4eb9-a4cf-1ce88c6e2237\") " pod="openshift-service-ca/service-ca-5fffc97bdf-pwwpf"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:05.571587    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-insights-runtime-extractor-tls podName:c41ceadb-e895-4b54-b3ea-88e0b6aa56d9 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:06.071570574 +0000 UTC m=+1.997793222 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hh6m5" (UID: "c41ceadb-e895-4b54-b3ea-88e0b6aa56d9") : secret "insights-runtime-extractor-tls" not found
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571600    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-run-openvswitch\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:05.571605    4969 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571613    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-log-socket\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:05.571641    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-cluster-monitoring-operator-tls podName:63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa nodeName:}" failed. No retries permitted until 2026-11-04 02:06:06.071629921 +0000 UTC m=+1.997852533 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-btjpg" (UID: "63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa") : secret "cluster-monitoring-operator-tls" not found
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571645    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-log-socket\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571655    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-sysctl-d\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571679    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-kubelet\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571696    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-registration-dir\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571711    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a395e26-3eac-4728-bbad-c42eaf6da56d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571721    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-kubelet\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571729    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a395e26-3eac-4728-bbad-c42eaf6da56d-system-cni-dir\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571746    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-sysctl-d\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571772    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a395e26-3eac-4728-bbad-c42eaf6da56d-system-cni-dir\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571775    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-registration-dir\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571781    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-var-lib-kubelet\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571807    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d-host-var-lib-kubelet\") pod \"multus-hclbr\" (UID: \"4a90ff43-d7f2-492d-98c9-31d9f6ca4b4d\") " pod="openshift-multus/multus-hclbr"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571812    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-device-dir\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571784    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a395e26-3eac-4728-bbad-c42eaf6da56d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jsh5m\" (UID: \"9a395e26-3eac-4728-bbad-c42eaf6da56d\") " pod="openshift-multus/multus-additional-cni-plugins-jsh5m"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571837    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-systemd\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571839    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-device-dir\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571873    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/16c89b3c-f63f-4489-8eae-5918677e1e61-host-slash\") pod \"iptables-alerter-qvk5m\" (UID: \"16c89b3c-f63f-4489-8eae-5918677e1e61\") " pod="openshift-network-operator/iptables-alerter-qvk5m"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571879    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/078e0741-51f6-4ae3-b9c0-bdf82a0f8f93-etc-systemd\") pod \"tuned-98zkj\" (UID: \"078e0741-51f6-4ae3-b9c0-bdf82a0f8f93\") " pod="openshift-cluster-node-tuning-operator/tuned-98zkj"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571900    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-run-ovn-kubernetes\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571906    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/16c89b3c-f63f-4489-8eae-5918677e1e61-host-slash\") pod \"iptables-alerter-qvk5m\" (UID: \"16c89b3c-f63f-4489-8eae-5918677e1e61\") " pod="openshift-network-operator/iptables-alerter-qvk5m"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571923    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-run-ovn-kubernetes\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571927    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571948    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.571995    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2l29t\" (UID: \"d7f9b7e3-b61b-4756-a01e-f5f026c5a3e1\") " pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:05.572006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.572004    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/94fedaf4-9e63-4c96-bb3e-e508815dd6c2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b7lwb\" (UID: \"94fedaf4-9e63-4c96-bb3e-e508815dd6c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b7lwb"
Nov 04 02:06:05.574374 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.572757    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3be2e6a4-4e32-4eb9-a4cf-1ce88c6e2237-signing-key\") pod \"service-ca-5fffc97bdf-pwwpf\" (UID: \"3be2e6a4-4e32-4eb9-a4cf-1ce88c6e2237\") " pod="openshift-service-ca/service-ca-5fffc97bdf-pwwpf"
Nov 04 02:06:05.616135 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.616117    4969 scope.go:122] "RemoveContainer" containerID="54ef74383bad59d1840a87bd68172bf3587b9c10deb740b68015579e57aeebae"
Nov 04 02:06:05.631358 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.631344    4969 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 02:06:05.633826 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.633809    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q8rb\" (UniqueName: \"kubernetes.io/projected/3be2e6a4-4e32-4eb9-a4cf-1ce88c6e2237-kube-api-access-2q8rb\") pod \"service-ca-5fffc97bdf-pwwpf\" (UID: \"3be2e6a4-4e32-4eb9-a4cf-1ce88c6e2237\") " pod="openshift-service-ca/service-ca-5fffc97bdf-pwwpf"
Nov 04 02:06:05.643410 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:05.643390    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-pwwpf"
Nov 04 02:06:06.007604 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:06.007567    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-pwwpf"]
Nov 04 02:06:06.011198 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:06.011167    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3be2e6a4_4e32_4eb9_a4cf_1ce88c6e2237.slice/crio-67ba188d3718551510158997911bd781599e8c4ad11de8c190893aeb8c547a9a WatchSource:0}: Error finding container 67ba188d3718551510158997911bd781599e8c4ad11de8c190893aeb8c547a9a: Status 404 returned error can't find the container with id 67ba188d3718551510158997911bd781599e8c4ad11de8c190893aeb8c547a9a
Nov 04 02:06:06.075628 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:06.075600    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pxvln\" (UID: \"0851f8eb-225e-4576-b5bc-dc6d72eefb6c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pxvln"
Nov 04 02:06:06.075721 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:06.075663    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-h72td\" (UID: \"3f0365d3-076c-4128-9349-b7dd71181e5f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"
Nov 04 02:06:06.075721 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:06.075695    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs\") pod \"network-metrics-daemon-4v7xm\" (UID: \"afe4b3d8-a18c-4587-9057-83123ffe3347\") " pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:06:06.075821 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:06.075732    4969 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 02:06:06.075821 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:06.075746    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:06:06.075821 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:06.075781    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:06:06.075821 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:06.075789    4969 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 04 02:06:06.075950 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:06.075796    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert podName:0851f8eb-225e-4576-b5bc-dc6d72eefb6c nodeName:}" failed. No retries permitted until 2026-11-04 02:06:07.075775406 +0000 UTC m=+3.001998020 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-pxvln" (UID: "0851f8eb-225e-4576-b5bc-dc6d72eefb6c") : secret "networking-console-plugin-cert" not found
Nov 04 02:06:06.075950 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:06.075845    4969 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 02:06:06.075950 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:06.075852    4969 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 02:06:06.075950 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:06.075862    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs podName:afe4b3d8-a18c-4587-9057-83123ffe3347 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:07.075845127 +0000 UTC m=+3.002067739 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs") pod "network-metrics-daemon-4v7xm" (UID: "afe4b3d8-a18c-4587-9057-83123ffe3347") : secret "metrics-daemon-secret" not found
Nov 04 02:06:06.075950 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:06.075873    4969 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 02:06:06.075950 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:06.075885    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls podName:3f0365d3-076c-4128-9349-b7dd71181e5f nodeName:}" failed. No retries permitted until 2026-11-04 02:06:07.075874759 +0000 UTC m=+3.002097386 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-h72td" (UID: "3f0365d3-076c-4128-9349-b7dd71181e5f") : secret "samples-operator-tls" not found
Nov 04 02:06:06.075950 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:06.075890    4969 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bd9d74c6b-thtzz: secret "image-registry-tls" not found
Nov 04 02:06:06.075950 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:06.075902    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:06:06.075950 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:06.075922    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls podName:cab3ee15-5d68-4840-83f4-ea8e2308e530 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:07.075911773 +0000 UTC m=+3.002134385 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls") pod "image-registry-5bd9d74c6b-thtzz" (UID: "cab3ee15-5d68-4840-83f4-ea8e2308e530") : secret "image-registry-tls" not found
Nov 04 02:06:06.075950 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:06.075944    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs podName:32699b90-bd7d-43f5-bdcc-53de9da6f244 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:07.075932934 +0000 UTC m=+3.002155545 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs") pod "router-default-6595f8bf85-dmj26" (UID: "32699b90-bd7d-43f5-bdcc-53de9da6f244") : secret "router-metrics-certs-default" not found
Nov 04 02:06:06.076261 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:06.075958    4969 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 02:06:06.076261 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:06.075966    4969 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bc84fd8c9-wbbgn: secret "image-registry-tls" not found
Nov 04 02:06:06.076261 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:06.075986    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls podName:12e9dc19-8f06-4b5d-8237-3ed5a449e09f nodeName:}" failed. No retries permitted until 2026-11-04 02:06:07.075980293 +0000 UTC m=+3.002202908 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls") pod "image-registry-6bc84fd8c9-wbbgn" (UID: "12e9dc19-8f06-4b5d-8237-3ed5a449e09f") : secret "image-registry-tls" not found
Nov 04 02:06:06.076261 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:06.075988    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:06:06.076261 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:06.076010    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:06.076261 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:06.076024    4969 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 02:06:06.076261 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:06.076027    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-btjpg\" (UID: \"63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg"
Nov 04 02:06:06.076261 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:06.076045    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls podName:63254ae3-0393-4094-9401-e18067d30080 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:07.07603708 +0000 UTC m=+3.002259690 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls") pod "dns-default-8sr5s" (UID: "63254ae3-0393-4094-9401-e18067d30080") : secret "dns-default-metrics-tls" not found
Nov 04 02:06:06.076261 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:06.076104    4969 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 02:06:06.076261 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:06.076124    4969 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 02:06:06.076261 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:06.076130    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32699b90-bd7d-43f5-bdcc-53de9da6f244-service-ca-bundle\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:06:06.076261 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:06.076142    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-insights-runtime-extractor-tls podName:c41ceadb-e895-4b54-b3ea-88e0b6aa56d9 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:07.076132486 +0000 UTC m=+3.002355326 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hh6m5" (UID: "c41ceadb-e895-4b54-b3ea-88e0b6aa56d9") : secret "insights-runtime-extractor-tls" not found
Nov 04 02:06:06.076261 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:06.076159    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-cluster-monitoring-operator-tls podName:63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa nodeName:}" failed. No retries permitted until 2026-11-04 02:06:07.076150774 +0000 UTC m=+3.002373387 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-btjpg" (UID: "63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa") : secret "cluster-monitoring-operator-tls" not found
Nov 04 02:06:06.076261 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:06.076241    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/32699b90-bd7d-43f5-bdcc-53de9da6f244-service-ca-bundle podName:32699b90-bd7d-43f5-bdcc-53de9da6f244 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:07.076224284 +0000 UTC m=+3.002446900 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/32699b90-bd7d-43f5-bdcc-53de9da6f244-service-ca-bundle") pod "router-default-6595f8bf85-dmj26" (UID: "32699b90-bd7d-43f5-bdcc-53de9da6f244") : configmap references non-existent config key: service-ca.crt
Nov 04 02:06:06.259069 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:06.259006    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-pwwpf" event={"ID":"3be2e6a4-4e32-4eb9-a4cf-1ce88c6e2237","Type":"ContainerStarted","Data":"41012f1b53c75bf9c64359e91b389695fb4a36fb32fcc2015416e69fcf205977"}
Nov 04 02:06:06.259069 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:06.259050    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-pwwpf" event={"ID":"3be2e6a4-4e32-4eb9-a4cf-1ce88c6e2237","Type":"ContainerStarted","Data":"67ba188d3718551510158997911bd781599e8c4ad11de8c190893aeb8c547a9a"}
Nov 04 02:06:06.260553 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:06.260533    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jcqmp_16f00163-6ec3-4c10-81e4-c8a501e15e82/console-operator/2.log"
Nov 04 02:06:06.260898 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:06.260886    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jcqmp_16f00163-6ec3-4c10-81e4-c8a501e15e82/console-operator/1.log"
Nov 04 02:06:06.260959 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:06.260915    4969 generic.go:356] "Generic (PLEG): container finished" podID="16f00163-6ec3-4c10-81e4-c8a501e15e82" containerID="6a884c945cc25c5e033018b77716c445c065bc85a68ab788d90b88594e84e036" exitCode=255
Nov 04 02:06:06.261014 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:06.260990    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp" event={"ID":"16f00163-6ec3-4c10-81e4-c8a501e15e82","Type":"ContainerDied","Data":"6a884c945cc25c5e033018b77716c445c065bc85a68ab788d90b88594e84e036"}
Nov 04 02:06:06.261053 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:06.261043    4969 scope.go:122] "RemoveContainer" containerID="54ef74383bad59d1840a87bd68172bf3587b9c10deb740b68015579e57aeebae"
Nov 04 02:06:06.261422 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:06.261405    4969 scope.go:122] "RemoveContainer" containerID="6a884c945cc25c5e033018b77716c445c065bc85a68ab788d90b88594e84e036"
Nov 04 02:06:06.261606 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:06.261584    4969 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-jcqmp_openshift-console-operator(16f00163-6ec3-4c10-81e4-c8a501e15e82)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp" podUID="16f00163-6ec3-4c10-81e4-c8a501e15e82"
Nov 04 02:06:06.466045 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:06.466013    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"
Nov 04 02:06:07.018221 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:07.018191    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:07.087737 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:07.087698    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-h72td\" (UID: \"3f0365d3-076c-4128-9349-b7dd71181e5f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"
Nov 04 02:06:07.087889 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:07.087748    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs\") pod \"network-metrics-daemon-4v7xm\" (UID: \"afe4b3d8-a18c-4587-9057-83123ffe3347\") " pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:06:07.087889 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:07.087795    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:06:07.087889 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:07.087825    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:06:07.087889 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:07.087851    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:06:07.088122 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:07.087919    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:06:07.088122 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:07.087951    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:07.088122 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:07.087979    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-btjpg\" (UID: \"63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg"
Nov 04 02:06:07.088269 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:07.088131    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32699b90-bd7d-43f5-bdcc-53de9da6f244-service-ca-bundle\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:06:07.088269 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:07.088162    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pxvln\" (UID: \"0851f8eb-225e-4576-b5bc-dc6d72eefb6c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pxvln"
Nov 04 02:06:07.088357 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:07.088292    4969 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 02:06:07.088411 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:07.088363    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert podName:0851f8eb-225e-4576-b5bc-dc6d72eefb6c nodeName:}" failed. No retries permitted until 2026-11-04 02:06:09.08834362 +0000 UTC m=+5.014566245 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-pxvln" (UID: "0851f8eb-225e-4576-b5bc-dc6d72eefb6c") : secret "networking-console-plugin-cert" not found
Nov 04 02:06:07.088472 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:07.088436    4969 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 02:06:07.088524 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:07.088472    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls podName:3f0365d3-076c-4128-9349-b7dd71181e5f nodeName:}" failed. No retries permitted until 2026-11-04 02:06:09.08845985 +0000 UTC m=+5.014682475 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-h72td" (UID: "3f0365d3-076c-4128-9349-b7dd71181e5f") : secret "samples-operator-tls" not found
Nov 04 02:06:07.088580 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:07.088533    4969 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 04 02:06:07.088580 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:07.088568    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs podName:afe4b3d8-a18c-4587-9057-83123ffe3347 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:09.08855759 +0000 UTC m=+5.014780206 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs") pod "network-metrics-daemon-4v7xm" (UID: "afe4b3d8-a18c-4587-9057-83123ffe3347") : secret "metrics-daemon-secret" not found
Nov 04 02:06:07.088886 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:07.088862    4969 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 02:06:07.088886 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:07.088880    4969 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bd9d74c6b-thtzz: secret "image-registry-tls" not found
Nov 04 02:06:07.089041 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:07.088915    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls podName:cab3ee15-5d68-4840-83f4-ea8e2308e530 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:09.088904681 +0000 UTC m=+5.015127305 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls") pod "image-registry-5bd9d74c6b-thtzz" (UID: "cab3ee15-5d68-4840-83f4-ea8e2308e530") : secret "image-registry-tls" not found
Nov 04 02:06:07.089041 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:07.088991    4969 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 02:06:07.089041 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:07.089017    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs podName:32699b90-bd7d-43f5-bdcc-53de9da6f244 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:09.089011418 +0000 UTC m=+5.015234042 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs") pod "router-default-6595f8bf85-dmj26" (UID: "32699b90-bd7d-43f5-bdcc-53de9da6f244") : secret "router-metrics-certs-default" not found
Nov 04 02:06:07.089172 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:07.089063    4969 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 04 02:06:07.089172 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:07.089070    4969 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6bc84fd8c9-wbbgn: secret "image-registry-tls" not found
Nov 04 02:06:07.089172 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:07.089087    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls podName:12e9dc19-8f06-4b5d-8237-3ed5a449e09f nodeName:}" failed. No retries permitted until 2026-11-04 02:06:09.089081774 +0000 UTC m=+5.015304384 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls") pod "image-registry-6bc84fd8c9-wbbgn" (UID: "12e9dc19-8f06-4b5d-8237-3ed5a449e09f") : secret "image-registry-tls" not found
Nov 04 02:06:07.089315 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:07.089208    4969 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 02:06:07.089315 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:07.089253    4969 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 02:06:07.089315 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:07.089277    4969 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 04 02:06:07.089315 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:07.089259    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/32699b90-bd7d-43f5-bdcc-53de9da6f244-service-ca-bundle podName:32699b90-bd7d-43f5-bdcc-53de9da6f244 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:09.089241499 +0000 UTC m=+5.015464125 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/32699b90-bd7d-43f5-bdcc-53de9da6f244-service-ca-bundle") pod "router-default-6595f8bf85-dmj26" (UID: "32699b90-bd7d-43f5-bdcc-53de9da6f244") : configmap references non-existent config key: service-ca.crt
Nov 04 02:06:07.089466 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:07.089341    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls podName:63254ae3-0393-4094-9401-e18067d30080 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:09.089322992 +0000 UTC m=+5.015545618 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls") pod "dns-default-8sr5s" (UID: "63254ae3-0393-4094-9401-e18067d30080") : secret "dns-default-metrics-tls" not found
Nov 04 02:06:07.089466 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:07.089360    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-insights-runtime-extractor-tls podName:c41ceadb-e895-4b54-b3ea-88e0b6aa56d9 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:09.089349477 +0000 UTC m=+5.015572095 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hh6m5" (UID: "c41ceadb-e895-4b54-b3ea-88e0b6aa56d9") : secret "insights-runtime-extractor-tls" not found
Nov 04 02:06:07.089466 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:07.089376    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-cluster-monitoring-operator-tls podName:63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa nodeName:}" failed. No retries permitted until 2026-11-04 02:06:09.089368026 +0000 UTC m=+5.015590640 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-btjpg" (UID: "63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa") : secret "cluster-monitoring-operator-tls" not found
Nov 04 02:06:07.112210 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:07.112187    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vjp7p_70aa0113-0423-4bdf-8335-d6c5bd707757/dns-node-resolver/0.log"
Nov 04 02:06:07.265788 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:07.265759    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jcqmp_16f00163-6ec3-4c10-81e4-c8a501e15e82/console-operator/2.log"
Nov 04 02:06:07.266198 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:07.266144    4969 scope.go:122] "RemoveContainer" containerID="6a884c945cc25c5e033018b77716c445c065bc85a68ab788d90b88594e84e036"
Nov 04 02:06:07.266368 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:07.266348    4969 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-jcqmp_openshift-console-operator(16f00163-6ec3-4c10-81e4-c8a501e15e82)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp" podUID="16f00163-6ec3-4c10-81e4-c8a501e15e82"
Nov 04 02:06:07.267358 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:07.267338    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nvl6s" event={"ID":"430bb96c-bf05-4a46-bdd3-612bd19a26d8","Type":"ContainerStarted","Data":"0c123070f55563cda7eb6709ff36a9bfada9412ef306bdbec07cad50c3e43906"}
Nov 04 02:06:07.911542 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:07.911475    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gzg65_8ddd2cf3-5cce-41a8-ad26-6946d3cb5910/node-ca/0.log"
Nov 04 02:06:08.011993 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:08.011933    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca/service-ca-5fffc97bdf-pwwpf" podStartSLOduration=5.01191699 podStartE2EDuration="5.01191699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:06:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:08.011705668 +0000 UTC m=+3.937928301" watchObservedRunningTime="2026-11-04 02:06:08.01191699 +0000 UTC m=+3.938139624"
Nov 04 02:06:08.255255 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:08.255187    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:06:08.276329 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:08.276300    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-dcf2j" event={"ID":"7c365e54-8c98-4271-a66e-9b13fe91afa7","Type":"ContainerStarted","Data":"bf0fea809f73eafdf933997590335c1602fd695ffc118750aafa0f5366eaf23e"}
Nov 04 02:06:08.311991 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:08.311969    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gflvk_301f5fdf-8bd3-49b3-be41-5684c210577c/serve-healthcheck-canary/0.log"
Nov 04 02:06:08.512134 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:08.512089    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-dcf2j_7c365e54-8c98-4271-a66e-9b13fe91afa7/migrator/0.log"
Nov 04 02:06:08.715609 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:08.715569    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-nvl6s_430bb96c-bf05-4a46-bdd3-612bd19a26d8/migrator/0.log"
Nov 04 02:06:08.912671 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:08.912600    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-96g6r_a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d/kube-storage-version-migrator-operator/0.log"
Nov 04 02:06:09.104973 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:09.104944    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:06:09.104973 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:09.104975    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:06:09.105178 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:09.104998    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:06:09.105178 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:09.105019    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:09.105178 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:09.105038    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-btjpg\" (UID: \"63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg"
Nov 04 02:06:09.105178 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:09.105106    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32699b90-bd7d-43f5-bdcc-53de9da6f244-service-ca-bundle\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:06:09.105178 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:09.105116    4969 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 04 02:06:09.105178 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:09.105125    4969 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 02:06:09.105178 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:09.105131    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pxvln\" (UID: \"0851f8eb-225e-4576-b5bc-dc6d72eefb6c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pxvln"
Nov 04 02:06:09.105178 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:09.105116    4969 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 02:06:09.105471 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:09.105186    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs podName:32699b90-bd7d-43f5-bdcc-53de9da6f244 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:13.10516859 +0000 UTC m=+9.031391218 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs") pod "router-default-6595f8bf85-dmj26" (UID: "32699b90-bd7d-43f5-bdcc-53de9da6f244") : secret "router-metrics-certs-default" not found
Nov 04 02:06:09.105471 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:09.105197    4969 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 04 02:06:09.105471 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:09.105250    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-insights-runtime-extractor-tls podName:c41ceadb-e895-4b54-b3ea-88e0b6aa56d9 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:13.105235683 +0000 UTC m=+9.031458294 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hh6m5" (UID: "c41ceadb-e895-4b54-b3ea-88e0b6aa56d9") : secret "insights-runtime-extractor-tls" not found
Nov 04 02:06:09.105471 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:09.105275    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls podName:63254ae3-0393-4094-9401-e18067d30080 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:13.105266234 +0000 UTC m=+9.031488847 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls") pod "dns-default-8sr5s" (UID: "63254ae3-0393-4094-9401-e18067d30080") : secret "dns-default-metrics-tls" not found
Nov 04 02:06:09.105471 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:09.105296    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert podName:0851f8eb-225e-4576-b5bc-dc6d72eefb6c nodeName:}" failed. No retries permitted until 2026-11-04 02:06:13.105287745 +0000 UTC m=+9.031510360 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-pxvln" (UID: "0851f8eb-225e-4576-b5bc-dc6d72eefb6c") : secret "networking-console-plugin-cert" not found
Nov 04 02:06:09.105471 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:09.105342    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-h72td\" (UID: \"3f0365d3-076c-4128-9349-b7dd71181e5f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"
Nov 04 02:06:09.105471 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:09.105371    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs\") pod \"network-metrics-daemon-4v7xm\" (UID: \"afe4b3d8-a18c-4587-9057-83123ffe3347\") " pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:06:09.105471 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:09.105405    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:06:09.105845 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:09.105483    4969 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 04 02:06:09.105845 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:09.105538    4969 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 04 02:06:09.105845 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:09.105545    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls podName:3f0365d3-076c-4128-9349-b7dd71181e5f nodeName:}" failed. No retries permitted until 2026-11-04 02:06:13.105530643 +0000 UTC m=+9.031753255 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-h72td" (UID: "3f0365d3-076c-4128-9349-b7dd71181e5f") : secret "samples-operator-tls" not found
Nov 04 02:06:09.105845 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:09.105597    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs podName:afe4b3d8-a18c-4587-9057-83123ffe3347 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:13.105580455 +0000 UTC m=+9.031803082 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs") pod "network-metrics-daemon-4v7xm" (UID: "afe4b3d8-a18c-4587-9057-83123ffe3347") : secret "metrics-daemon-secret" not found
Nov 04 02:06:09.105845 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:09.105804    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32699b90-bd7d-43f5-bdcc-53de9da6f244-service-ca-bundle\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:06:09.107459 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:09.107435    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls\") pod \"image-registry-6bc84fd8c9-wbbgn\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") " pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:06:09.107551 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:09.107468    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-btjpg\" (UID: \"63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg"
Nov 04 02:06:09.107603 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:09.107589    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls\") pod \"image-registry-5bd9d74c6b-thtzz\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") " pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:06:09.213994 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:09.213934    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg"
Nov 04 02:06:09.214157 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:09.214142    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:06:09.224583 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:09.224565    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:06:09.692473 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:09.692441    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg"]
Nov 04 02:06:09.695405 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:09.695372    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63f7e9d4_d5e9_4115_9aa8_fc8a6aae5faa.slice/crio-45374889efe7a385115c1fb8ca0205abf20bf61600d59d913ca74d3316fb34c2 WatchSource:0}: Error finding container 45374889efe7a385115c1fb8ca0205abf20bf61600d59d913ca74d3316fb34c2: Status 404 returned error can't find the container with id 45374889efe7a385115c1fb8ca0205abf20bf61600d59d913ca74d3316fb34c2
Nov 04 02:06:09.696207 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:09.696187    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bd9d74c6b-thtzz"]
Nov 04 02:06:09.698891 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:09.698869    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcab3ee15_5d68_4840_83f4_ea8e2308e530.slice/crio-f183641efc5ebde82c8a28e5630870e3933a7fb21a6bb12f5fa3089bfcb85b1e WatchSource:0}: Error finding container f183641efc5ebde82c8a28e5630870e3933a7fb21a6bb12f5fa3089bfcb85b1e: Status 404 returned error can't find the container with id f183641efc5ebde82c8a28e5630870e3933a7fb21a6bb12f5fa3089bfcb85b1e
Nov 04 02:06:09.746180 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:09.746131    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"]
Nov 04 02:06:09.746256 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:09.746189    4969 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"
Nov 04 02:06:09.746548 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:09.746520    4969 scope.go:122] "RemoveContainer" containerID="6a884c945cc25c5e033018b77716c445c065bc85a68ab788d90b88594e84e036"
Nov 04 02:06:09.746705 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:09.746682    4969 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-jcqmp_openshift-console-operator(16f00163-6ec3-4c10-81e4-c8a501e15e82)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp" podUID="16f00163-6ec3-4c10-81e4-c8a501e15e82"
Nov 04 02:06:09.748322 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:09.748296    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12e9dc19_8f06_4b5d_8237_3ed5a449e09f.slice/crio-41d25760db43fb80c551b6bcbd2dffc15a34cc035bb728cf3f4d5ddef0db1d2a WatchSource:0}: Error finding container 41d25760db43fb80c551b6bcbd2dffc15a34cc035bb728cf3f4d5ddef0db1d2a: Status 404 returned error can't find the container with id 41d25760db43fb80c551b6bcbd2dffc15a34cc035bb728cf3f4d5ddef0db1d2a
Nov 04 02:06:10.287705 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:10.287669    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg" event={"ID":"63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa","Type":"ContainerStarted","Data":"45374889efe7a385115c1fb8ca0205abf20bf61600d59d913ca74d3316fb34c2"}
Nov 04 02:06:10.289243 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:10.289213    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz" event={"ID":"cab3ee15-5d68-4840-83f4-ea8e2308e530","Type":"ContainerStarted","Data":"5ea586b54622e0bfec7ef91bec4922b7f90488d84f50b5638568da3f04a402fe"}
Nov 04 02:06:10.289616 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:10.289254    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz" event={"ID":"cab3ee15-5d68-4840-83f4-ea8e2308e530","Type":"ContainerStarted","Data":"f183641efc5ebde82c8a28e5630870e3933a7fb21a6bb12f5fa3089bfcb85b1e"}
Nov 04 02:06:10.290744 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:10.290717    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn" event={"ID":"12e9dc19-8f06-4b5d-8237-3ed5a449e09f","Type":"ContainerStarted","Data":"3301eed52149903e3de804984c04647f3a629fd88b0e0e99f5851b47131edd01"}
Nov 04 02:06:10.290847 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:10.290765    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn" event={"ID":"12e9dc19-8f06-4b5d-8237-3ed5a449e09f","Type":"ContainerStarted","Data":"41d25760db43fb80c551b6bcbd2dffc15a34cc035bb728cf3f4d5ddef0db1d2a"}
Nov 04 02:06:10.290847 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:10.290792    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:06:10.757116 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:10.757035    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-928nm"
Nov 04 02:06:11.294702 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:11.294675    4969 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 02:06:12.299491 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:12.299462    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-btjpg" event={"ID":"63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa","Type":"ContainerStarted","Data":"9175615f6be82ba36e9e23e4b14358a688fbba2fcf49143ca4c086b1b9d2b274"}
Nov 04 02:06:12.672787 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:12.671447    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:06:13.145986 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:13.145951    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pxvln\" (UID: \"0851f8eb-225e-4576-b5bc-dc6d72eefb6c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pxvln"
Nov 04 02:06:13.146182 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:13.145993    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-h72td\" (UID: \"3f0365d3-076c-4128-9349-b7dd71181e5f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"
Nov 04 02:06:13.146182 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:13.146010    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs\") pod \"network-metrics-daemon-4v7xm\" (UID: \"afe4b3d8-a18c-4587-9057-83123ffe3347\") " pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:06:13.146182 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:13.146034    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:06:13.146182 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:13.146131    4969 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 04 02:06:13.146471 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:13.146191    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs podName:afe4b3d8-a18c-4587-9057-83123ffe3347 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:21.14617528 +0000 UTC m=+17.072397936 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs") pod "network-metrics-daemon-4v7xm" (UID: "afe4b3d8-a18c-4587-9057-83123ffe3347") : secret "metrics-daemon-secret" not found
Nov 04 02:06:13.146471 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:13.146225    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:06:13.146471 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:13.146289    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:13.146471 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:13.146353    4969 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 02:06:13.146471 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:13.146400    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls podName:63254ae3-0393-4094-9401-e18067d30080 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:21.146387543 +0000 UTC m=+17.072610160 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls") pod "dns-default-8sr5s" (UID: "63254ae3-0393-4094-9401-e18067d30080") : secret "dns-default-metrics-tls" not found
Nov 04 02:06:13.146471 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:13.146411    4969 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 02:06:13.146471 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:13.146477    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-insights-runtime-extractor-tls podName:c41ceadb-e895-4b54-b3ea-88e0b6aa56d9 nodeName:}" failed. No retries permitted until 2026-11-04 02:06:21.146462588 +0000 UTC m=+17.072685220 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hh6m5" (UID: "c41ceadb-e895-4b54-b3ea-88e0b6aa56d9") : secret "insights-runtime-extractor-tls" not found
Nov 04 02:06:13.148528 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:13.148500    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0851f8eb-225e-4576-b5bc-dc6d72eefb6c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-pxvln\" (UID: \"0851f8eb-225e-4576-b5bc-dc6d72eefb6c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-pxvln"
Nov 04 02:06:13.148620 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:13.148536    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32699b90-bd7d-43f5-bdcc-53de9da6f244-metrics-certs\") pod \"router-default-6595f8bf85-dmj26\" (UID: \"32699b90-bd7d-43f5-bdcc-53de9da6f244\") " pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:06:13.148620 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:13.148566    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f0365d3-076c-4128-9349-b7dd71181e5f-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-h72td\" (UID: \"3f0365d3-076c-4128-9349-b7dd71181e5f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"
Nov 04 02:06:13.413384 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:13.413323    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:06:13.414837 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:13.414818    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"
Nov 04 02:06:13.417286 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:13.417264    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-pxvln"
Nov 04 02:06:13.800247 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:13.800214    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-pxvln"]
Nov 04 02:06:13.841666 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:13.841644    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0851f8eb_225e_4576_b5bc_dc6d72eefb6c.slice/crio-3c96887d1b6ab8cf9a5f5a71b66a8d3b7856a4d9068ae8f73075ae3922af5a61 WatchSource:0}: Error finding container 3c96887d1b6ab8cf9a5f5a71b66a8d3b7856a4d9068ae8f73075ae3922af5a61: Status 404 returned error can't find the container with id 3c96887d1b6ab8cf9a5f5a71b66a8d3b7856a4d9068ae8f73075ae3922af5a61
Nov 04 02:06:13.854980 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:13.854943    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6595f8bf85-dmj26"]
Nov 04 02:06:13.857385 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:13.857361    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32699b90_bd7d_43f5_bdcc_53de9da6f244.slice/crio-ffedb88f536c6a6c7c4fd1c0d5d63acc2a099731c6b823d86cbb5ef344a7211c WatchSource:0}: Error finding container ffedb88f536c6a6c7c4fd1c0d5d63acc2a099731c6b823d86cbb5ef344a7211c: Status 404 returned error can't find the container with id ffedb88f536c6a6c7c4fd1c0d5d63acc2a099731c6b823d86cbb5ef344a7211c
Nov 04 02:06:13.900421 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:13.900329    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td"]
Nov 04 02:06:14.274292 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:14.274263    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:14.274501 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:14.274489    4969 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 02:06:14.307524 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:14.307496    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td" event={"ID":"3f0365d3-076c-4128-9349-b7dd71181e5f","Type":"ContainerStarted","Data":"4b5e1d762093ff475fcac54d11a284c289f110df8a982c6060581e362787389b"}
Nov 04 02:06:14.309026 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:14.308998    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6595f8bf85-dmj26" event={"ID":"32699b90-bd7d-43f5-bdcc-53de9da6f244","Type":"ContainerStarted","Data":"191c3db35b3f69eda443c09b483c1db508f9ed88cfe9cc36b5999e11549f6bbe"}
Nov 04 02:06:14.309194 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:14.309042    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6595f8bf85-dmj26" event={"ID":"32699b90-bd7d-43f5-bdcc-53de9da6f244","Type":"ContainerStarted","Data":"ffedb88f536c6a6c7c4fd1c0d5d63acc2a099731c6b823d86cbb5ef344a7211c"}
Nov 04 02:06:14.310135 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:14.310084    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-pxvln" event={"ID":"0851f8eb-225e-4576-b5bc-dc6d72eefb6c","Type":"ContainerStarted","Data":"3c96887d1b6ab8cf9a5f5a71b66a8d3b7856a4d9068ae8f73075ae3922af5a61"}
Nov 04 02:06:14.414442 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:14.414416    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:06:14.417008 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:14.416987    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:06:14.738594 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:14.738535    4969 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-63.us-west-1.compute.internal" event="NodeReady"
Nov 04 02:06:14.903807 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:14.903775    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:15.131114 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:15.131066    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:15.231797 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:15.231768    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:06:15.233270 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:15.233244    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6595f8bf85-dmj26"
Nov 04 02:06:15.505187 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:15.505162    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:15.708702 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:15.708673    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:15.961127 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:15.961074    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:16.147606 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:16.145546    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:16.320541 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:16.320505    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td" event={"ID":"3f0365d3-076c-4128-9349-b7dd71181e5f","Type":"ContainerStarted","Data":"75f3b336e2c19ab59f75cb0d051726b407b3d9e5c1f7a2c10bb100067363cdcc"}
Nov 04 02:06:16.320668 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:16.320572    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-h72td" event={"ID":"3f0365d3-076c-4128-9349-b7dd71181e5f","Type":"ContainerStarted","Data":"41fe553c67e3860a37276ecbe8c771aa31242957348c790eb76b2a7073b44faf"}
Nov 04 02:06:16.321966 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:16.321935    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-pxvln" event={"ID":"0851f8eb-225e-4576-b5bc-dc6d72eefb6c","Type":"ContainerStarted","Data":"6801af769efdc2cf98c92f53273a8a445ff5d334cd52d83df3af93da3b1a5378"}
Nov 04 02:06:16.326116 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:16.326074    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:16.711035 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:16.710997    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:17.308424 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:17.308393    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:17.899395 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:17.899361    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:18.112562 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:18.112529    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:18.308748 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:18.308718    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:18.509237 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:18.509211    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:19.194875 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:19.194841    4969 scope.go:122] "RemoveContainer" containerID="6a884c945cc25c5e033018b77716c445c065bc85a68ab788d90b88594e84e036"
Nov 04 02:06:19.333410 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:19.333385    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jcqmp_16f00163-6ec3-4c10-81e4-c8a501e15e82/console-operator/2.log"
Nov 04 02:06:19.333554 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:19.333476    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp" event={"ID":"16f00163-6ec3-4c10-81e4-c8a501e15e82","Type":"ContainerStarted","Data":"51420e4455f623ca99f0cb9ca88500a21ceb9d37b337041f208b07c93cf07a69"}
Nov 04 02:06:19.333746 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:19.333725    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"
Nov 04 02:06:19.334577 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:19.334555    4969 patch_prober.go:29] interesting pod/console-operator-7cd56b8dbc-jcqmp container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.14:8443/readyz\": dial tcp 10.132.0.14:8443: connect: connection refused" start-of-body=
Nov 04 02:06:19.334698 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:19.334595    4969 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp" podUID="16f00163-6ec3-4c10-81e4-c8a501e15e82" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.14:8443/readyz\": dial tcp 10.132.0.14:8443: connect: connection refused"
Nov 04 02:06:20.341276 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:20.341248    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-jcqmp"
Nov 04 02:06:20.719228 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:20.719152    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:20.895860 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:20.895824    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:21.073954 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:21.073926    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:21.218085 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:21.218051    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs\") pod \"network-metrics-daemon-4v7xm\" (UID: \"afe4b3d8-a18c-4587-9057-83123ffe3347\") " pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:06:21.218266 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:21.218121    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:06:21.218266 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:21.218155    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:21.220638 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:21.220611    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/63254ae3-0393-4094-9401-e18067d30080-metrics-tls\") pod \"dns-default-8sr5s\" (UID: \"63254ae3-0393-4094-9401-e18067d30080\") " pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:06:21.220738 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:21.220645    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c41ceadb-e895-4b54-b3ea-88e0b6aa56d9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hh6m5\" (UID: \"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9\") " pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:21.220738 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:21.220658    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afe4b3d8-a18c-4587-9057-83123ffe3347-metrics-certs\") pod \"network-metrics-daemon-4v7xm\" (UID: \"afe4b3d8-a18c-4587-9057-83123ffe3347\") " pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:06:21.225554 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:21.225536    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:06:21.231142 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:21.231122    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hh6m5"
Nov 04 02:06:21.252267 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:21.252211    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:21.518270 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:21.518248    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4v7xm"
Nov 04 02:06:21.595230 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:21.595207    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8sr5s"]
Nov 04 02:06:21.634483 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:21.634454    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63254ae3_0393_4094_9401_e18067d30080.slice/crio-b4572e904ec98011673c4171cf9c6af93ab0ea976321076d934435fd3972c974 WatchSource:0}: Error finding container b4572e904ec98011673c4171cf9c6af93ab0ea976321076d934435fd3972c974: Status 404 returned error can't find the container with id b4572e904ec98011673c4171cf9c6af93ab0ea976321076d934435fd3972c974
Nov 04 02:06:21.644866 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:21.644843    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hh6m5"]
Nov 04 02:06:21.648202 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:21.648182    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc41ceadb_e895_4b54_b3ea_88e0b6aa56d9.slice/crio-6b062bc613831a982554569496776f7a69a0585464a8fb98c3f2c56b334caad9 WatchSource:0}: Error finding container 6b062bc613831a982554569496776f7a69a0585464a8fb98c3f2c56b334caad9: Status 404 returned error can't find the container with id 6b062bc613831a982554569496776f7a69a0585464a8fb98c3f2c56b334caad9
Nov 04 02:06:21.688680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:21.688658    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2l29t"
Nov 04 02:06:21.888682 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:21.888657    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafe4b3d8_a18c_4587_9057_83123ffe3347.slice/crio-571eb2be76cdce1b7a4768c92487134b588c6beb98b9572d01b17d8c3d4cf31c WatchSource:0}: Error finding container 571eb2be76cdce1b7a4768c92487134b588c6beb98b9572d01b17d8c3d4cf31c: Status 404 returned error can't find the container with id 571eb2be76cdce1b7a4768c92487134b588c6beb98b9572d01b17d8c3d4cf31c
Nov 04 02:06:21.892257 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:21.892239    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4v7xm"]
Nov 04 02:06:22.350266 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.350227    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4v7xm" event={"ID":"afe4b3d8-a18c-4587-9057-83123ffe3347","Type":"ContainerStarted","Data":"571eb2be76cdce1b7a4768c92487134b588c6beb98b9572d01b17d8c3d4cf31c"}
Nov 04 02:06:22.351405 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.351379    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8sr5s" event={"ID":"63254ae3-0393-4094-9401-e18067d30080","Type":"ContainerStarted","Data":"b4572e904ec98011673c4171cf9c6af93ab0ea976321076d934435fd3972c974"}
Nov 04 02:06:22.352893 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.352868    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hh6m5" event={"ID":"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9","Type":"ContainerStarted","Data":"01092be3f024791e5c292a51647b4448365e8da98aa75b66742e3dbdeb16e4cc"}
Nov 04 02:06:22.352986 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.352905    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hh6m5" event={"ID":"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9","Type":"ContainerStarted","Data":"6b062bc613831a982554569496776f7a69a0585464a8fb98c3f2c56b334caad9"}
Nov 04 02:06:22.701182 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.701082    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5bd9d74c6b-thtzz"]
Nov 04 02:06:22.769289 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.769257    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-xw9tf"]
Nov 04 02:06:22.771490 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.771453    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-xw9tf"
Nov 04 02:06:22.807569 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.807542    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-2qtvl"]
Nov 04 02:06:22.809921 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.809900    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-76f8d66774-gl6nr"]
Nov 04 02:06:22.810037 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.810020    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-2qtvl"
Nov 04 02:06:22.811717 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.811696    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76f8d66774-gl6nr"
Nov 04 02:06:22.827763 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.827745    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-5nxcc\""
Nov 04 02:06:22.827879 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.827827    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Nov 04 02:06:22.833116 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.833081    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/48814638-7f4b-4276-9d0d-97b0d80f46cd-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-xw9tf\" (UID: \"48814638-7f4b-4276-9d0d-97b0d80f46cd\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-xw9tf"
Nov 04 02:06:22.862981 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.862942    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-xw9tf"]
Nov 04 02:06:22.864748 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.864728    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-2qtvl"]
Nov 04 02:06:22.868011 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.867991    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76f8d66774-gl6nr"]
Nov 04 02:06:22.871192 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.870354    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Nov 04 02:06:22.871192 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.870478    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Nov 04 02:06:22.927781 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.927763    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-bzrp8\""
Nov 04 02:06:22.934142 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.934120    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6245edcf-e1f3-4dfb-84ef-a2911f854ae4-image-registry-private-configuration\") pod \"image-registry-76f8d66774-gl6nr\" (UID: \"6245edcf-e1f3-4dfb-84ef-a2911f854ae4\") " pod="openshift-image-registry/image-registry-76f8d66774-gl6nr"
Nov 04 02:06:22.934227 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.934154    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98621f1f-0f97-44be-995b-9a37a0217747-tmp\") pod \"downloads-6dd789c8b7-2qtvl\" (UID: \"98621f1f-0f97-44be-995b-9a37a0217747\") " pod="openshift-console/downloads-6dd789c8b7-2qtvl"
Nov 04 02:06:22.934227 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.934180    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/48814638-7f4b-4276-9d0d-97b0d80f46cd-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-xw9tf\" (UID: \"48814638-7f4b-4276-9d0d-97b0d80f46cd\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-xw9tf"
Nov 04 02:06:22.934293 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.934232    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6245edcf-e1f3-4dfb-84ef-a2911f854ae4-bound-sa-token\") pod \"image-registry-76f8d66774-gl6nr\" (UID: \"6245edcf-e1f3-4dfb-84ef-a2911f854ae4\") " pod="openshift-image-registry/image-registry-76f8d66774-gl6nr"
Nov 04 02:06:22.934293 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.934279    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6245edcf-e1f3-4dfb-84ef-a2911f854ae4-registry-certificates\") pod \"image-registry-76f8d66774-gl6nr\" (UID: \"6245edcf-e1f3-4dfb-84ef-a2911f854ae4\") " pod="openshift-image-registry/image-registry-76f8d66774-gl6nr"
Nov 04 02:06:22.934435 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.934423    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6srgr\" (UniqueName: \"kubernetes.io/projected/6245edcf-e1f3-4dfb-84ef-a2911f854ae4-kube-api-access-6srgr\") pod \"image-registry-76f8d66774-gl6nr\" (UID: \"6245edcf-e1f3-4dfb-84ef-a2911f854ae4\") " pod="openshift-image-registry/image-registry-76f8d66774-gl6nr"
Nov 04 02:06:22.934466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.934454    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6245edcf-e1f3-4dfb-84ef-a2911f854ae4-ca-trust-extracted\") pod \"image-registry-76f8d66774-gl6nr\" (UID: \"6245edcf-e1f3-4dfb-84ef-a2911f854ae4\") " pod="openshift-image-registry/image-registry-76f8d66774-gl6nr"
Nov 04 02:06:22.934498 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.934470    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6245edcf-e1f3-4dfb-84ef-a2911f854ae4-trusted-ca\") pod \"image-registry-76f8d66774-gl6nr\" (UID: \"6245edcf-e1f3-4dfb-84ef-a2911f854ae4\") " pod="openshift-image-registry/image-registry-76f8d66774-gl6nr"
Nov 04 02:06:22.934498 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.934484    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6245edcf-e1f3-4dfb-84ef-a2911f854ae4-installation-pull-secrets\") pod \"image-registry-76f8d66774-gl6nr\" (UID: \"6245edcf-e1f3-4dfb-84ef-a2911f854ae4\") " pod="openshift-image-registry/image-registry-76f8d66774-gl6nr"
Nov 04 02:06:22.934564 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.934530    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7hpb\" (UniqueName: \"kubernetes.io/projected/98621f1f-0f97-44be-995b-9a37a0217747-kube-api-access-t7hpb\") pod \"downloads-6dd789c8b7-2qtvl\" (UID: \"98621f1f-0f97-44be-995b-9a37a0217747\") " pod="openshift-console/downloads-6dd789c8b7-2qtvl"
Nov 04 02:06:22.934664 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.934642    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6245edcf-e1f3-4dfb-84ef-a2911f854ae4-registry-tls\") pod \"image-registry-76f8d66774-gl6nr\" (UID: \"6245edcf-e1f3-4dfb-84ef-a2911f854ae4\") " pod="openshift-image-registry/image-registry-76f8d66774-gl6nr"
Nov 04 02:06:22.936484 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:22.936459    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/48814638-7f4b-4276-9d0d-97b0d80f46cd-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-xw9tf\" (UID: \"48814638-7f4b-4276-9d0d-97b0d80f46cd\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-xw9tf"
Nov 04 02:06:23.006250 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.006223    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:23.035373 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.035334    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6245edcf-e1f3-4dfb-84ef-a2911f854ae4-bound-sa-token\") pod \"image-registry-76f8d66774-gl6nr\" (UID: \"6245edcf-e1f3-4dfb-84ef-a2911f854ae4\") " pod="openshift-image-registry/image-registry-76f8d66774-gl6nr"
Nov 04 02:06:23.035507 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.035380    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6245edcf-e1f3-4dfb-84ef-a2911f854ae4-registry-certificates\") pod \"image-registry-76f8d66774-gl6nr\" (UID: \"6245edcf-e1f3-4dfb-84ef-a2911f854ae4\") " pod="openshift-image-registry/image-registry-76f8d66774-gl6nr"
Nov 04 02:06:23.035507 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.035407    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6srgr\" (UniqueName: \"kubernetes.io/projected/6245edcf-e1f3-4dfb-84ef-a2911f854ae4-kube-api-access-6srgr\") pod \"image-registry-76f8d66774-gl6nr\" (UID: \"6245edcf-e1f3-4dfb-84ef-a2911f854ae4\") " pod="openshift-image-registry/image-registry-76f8d66774-gl6nr"
Nov 04 02:06:23.035507 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.035438    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6245edcf-e1f3-4dfb-84ef-a2911f854ae4-ca-trust-extracted\") pod \"image-registry-76f8d66774-gl6nr\" (UID: \"6245edcf-e1f3-4dfb-84ef-a2911f854ae4\") " pod="openshift-image-registry/image-registry-76f8d66774-gl6nr"
Nov 04 02:06:23.035507 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.035464    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6245edcf-e1f3-4dfb-84ef-a2911f854ae4-trusted-ca\") pod \"image-registry-76f8d66774-gl6nr\" (UID: \"6245edcf-e1f3-4dfb-84ef-a2911f854ae4\") " pod="openshift-image-registry/image-registry-76f8d66774-gl6nr"
Nov 04 02:06:23.035507 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.035481    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6245edcf-e1f3-4dfb-84ef-a2911f854ae4-installation-pull-secrets\") pod \"image-registry-76f8d66774-gl6nr\" (UID: \"6245edcf-e1f3-4dfb-84ef-a2911f854ae4\") " pod="openshift-image-registry/image-registry-76f8d66774-gl6nr"
Nov 04 02:06:23.035507 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.035501    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7hpb\" (UniqueName: \"kubernetes.io/projected/98621f1f-0f97-44be-995b-9a37a0217747-kube-api-access-t7hpb\") pod \"downloads-6dd789c8b7-2qtvl\" (UID: \"98621f1f-0f97-44be-995b-9a37a0217747\") " pod="openshift-console/downloads-6dd789c8b7-2qtvl"
Nov 04 02:06:23.035807 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.035532    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6245edcf-e1f3-4dfb-84ef-a2911f854ae4-registry-tls\") pod \"image-registry-76f8d66774-gl6nr\" (UID: \"6245edcf-e1f3-4dfb-84ef-a2911f854ae4\") " pod="openshift-image-registry/image-registry-76f8d66774-gl6nr"
Nov 04 02:06:23.036151 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.036088    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6245edcf-e1f3-4dfb-84ef-a2911f854ae4-image-registry-private-configuration\") pod \"image-registry-76f8d66774-gl6nr\" (UID: \"6245edcf-e1f3-4dfb-84ef-a2911f854ae4\") " pod="openshift-image-registry/image-registry-76f8d66774-gl6nr"
Nov 04 02:06:23.036241 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.036193    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98621f1f-0f97-44be-995b-9a37a0217747-tmp\") pod \"downloads-6dd789c8b7-2qtvl\" (UID: \"98621f1f-0f97-44be-995b-9a37a0217747\") " pod="openshift-console/downloads-6dd789c8b7-2qtvl"
Nov 04 02:06:23.036308 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.036255    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6245edcf-e1f3-4dfb-84ef-a2911f854ae4-ca-trust-extracted\") pod \"image-registry-76f8d66774-gl6nr\" (UID: \"6245edcf-e1f3-4dfb-84ef-a2911f854ae4\") " pod="openshift-image-registry/image-registry-76f8d66774-gl6nr"
Nov 04 02:06:23.036494 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.036463    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6245edcf-e1f3-4dfb-84ef-a2911f854ae4-registry-certificates\") pod \"image-registry-76f8d66774-gl6nr\" (UID: \"6245edcf-e1f3-4dfb-84ef-a2911f854ae4\") " pod="openshift-image-registry/image-registry-76f8d66774-gl6nr"
Nov 04 02:06:23.036603 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.036571    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98621f1f-0f97-44be-995b-9a37a0217747-tmp\") pod \"downloads-6dd789c8b7-2qtvl\" (UID: \"98621f1f-0f97-44be-995b-9a37a0217747\") " pod="openshift-console/downloads-6dd789c8b7-2qtvl"
Nov 04 02:06:23.036687 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.036663    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6245edcf-e1f3-4dfb-84ef-a2911f854ae4-trusted-ca\") pod \"image-registry-76f8d66774-gl6nr\" (UID: \"6245edcf-e1f3-4dfb-84ef-a2911f854ae4\") " pod="openshift-image-registry/image-registry-76f8d66774-gl6nr"
Nov 04 02:06:23.038421 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.038396    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6245edcf-e1f3-4dfb-84ef-a2911f854ae4-installation-pull-secrets\") pod \"image-registry-76f8d66774-gl6nr\" (UID: \"6245edcf-e1f3-4dfb-84ef-a2911f854ae4\") " pod="openshift-image-registry/image-registry-76f8d66774-gl6nr"
Nov 04 02:06:23.038513 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.038408    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6245edcf-e1f3-4dfb-84ef-a2911f854ae4-registry-tls\") pod \"image-registry-76f8d66774-gl6nr\" (UID: \"6245edcf-e1f3-4dfb-84ef-a2911f854ae4\") " pod="openshift-image-registry/image-registry-76f8d66774-gl6nr"
Nov 04 02:06:23.038628 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.038601    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6245edcf-e1f3-4dfb-84ef-a2911f854ae4-image-registry-private-configuration\") pod \"image-registry-76f8d66774-gl6nr\" (UID: \"6245edcf-e1f3-4dfb-84ef-a2911f854ae4\") " pod="openshift-image-registry/image-registry-76f8d66774-gl6nr"
Nov 04 02:06:23.085056 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.085031    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-xw9tf"
Nov 04 02:06:23.099103 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.099071    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7hpb\" (UniqueName: \"kubernetes.io/projected/98621f1f-0f97-44be-995b-9a37a0217747-kube-api-access-t7hpb\") pod \"downloads-6dd789c8b7-2qtvl\" (UID: \"98621f1f-0f97-44be-995b-9a37a0217747\") " pod="openshift-console/downloads-6dd789c8b7-2qtvl"
Nov 04 02:06:23.099304 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.099287    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6245edcf-e1f3-4dfb-84ef-a2911f854ae4-bound-sa-token\") pod \"image-registry-76f8d66774-gl6nr\" (UID: \"6245edcf-e1f3-4dfb-84ef-a2911f854ae4\") " pod="openshift-image-registry/image-registry-76f8d66774-gl6nr"
Nov 04 02:06:23.099912 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.099897    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6srgr\" (UniqueName: \"kubernetes.io/projected/6245edcf-e1f3-4dfb-84ef-a2911f854ae4-kube-api-access-6srgr\") pod \"image-registry-76f8d66774-gl6nr\" (UID: \"6245edcf-e1f3-4dfb-84ef-a2911f854ae4\") " pod="openshift-image-registry/image-registry-76f8d66774-gl6nr"
Nov 04 02:06:23.122912 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.122884    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-2qtvl"
Nov 04 02:06:23.129081 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.129060    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76f8d66774-gl6nr"
Nov 04 02:06:23.182073 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.182037    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:23.359911 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.359875    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hh6m5" event={"ID":"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9","Type":"ContainerStarted","Data":"bd180516248c496801ee7e7629c102e63a0f406bc09ddf8534ec64c8ac2eff90"}
Nov 04 02:06:23.458876 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.458839    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-xw9tf"]
Nov 04 02:06:23.463652 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:23.463622    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48814638_7f4b_4276_9d0d_97b0d80f46cd.slice/crio-f0cfaffbbc54ee087185db929cad485d3ec495b4d7f5fa0f044125c4a23a6a41 WatchSource:0}: Error finding container f0cfaffbbc54ee087185db929cad485d3ec495b4d7f5fa0f044125c4a23a6a41: Status 404 returned error can't find the container with id f0cfaffbbc54ee087185db929cad485d3ec495b4d7f5fa0f044125c4a23a6a41
Nov 04 02:06:23.472996 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.472968    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:23.518541 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.518511    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76f8d66774-gl6nr"]
Nov 04 02:06:23.527197 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:23.527161    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6245edcf_e1f3_4dfb_84ef_a2911f854ae4.slice/crio-59c760dc278a26051ad0df783029b3ac296c09349cecd2539867ccf6df1fc80a WatchSource:0}: Error finding container 59c760dc278a26051ad0df783029b3ac296c09349cecd2539867ccf6df1fc80a: Status 404 returned error can't find the container with id 59c760dc278a26051ad0df783029b3ac296c09349cecd2539867ccf6df1fc80a
Nov 04 02:06:23.563804 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.563779    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-2qtvl"]
Nov 04 02:06:23.572303 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:23.572254    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98621f1f_0f97_44be_995b_9a37a0217747.slice/crio-1800f50a24b9b6ba564258fa78b65e8c564cb38e30ef5be07208e9107ad48127 WatchSource:0}: Error finding container 1800f50a24b9b6ba564258fa78b65e8c564cb38e30ef5be07208e9107ad48127: Status 404 returned error can't find the container with id 1800f50a24b9b6ba564258fa78b65e8c564cb38e30ef5be07208e9107ad48127
Nov 04 02:06:23.659536 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:23.659453    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:24.366417 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:24.366377    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-2qtvl" event={"ID":"98621f1f-0f97-44be-995b-9a37a0217747","Type":"ContainerStarted","Data":"1800f50a24b9b6ba564258fa78b65e8c564cb38e30ef5be07208e9107ad48127"}
Nov 04 02:06:24.367789 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:24.367758    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-xw9tf" event={"ID":"48814638-7f4b-4276-9d0d-97b0d80f46cd","Type":"ContainerStarted","Data":"f0cfaffbbc54ee087185db929cad485d3ec495b4d7f5fa0f044125c4a23a6a41"}
Nov 04 02:06:24.369957 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:24.369927    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76f8d66774-gl6nr" event={"ID":"6245edcf-e1f3-4dfb-84ef-a2911f854ae4","Type":"ContainerStarted","Data":"db89cebb9287770fb7a33ded92078bcef14fe2f5e98139fb02062f3f00fc9b4d"}
Nov 04 02:06:24.370082 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:24.369973    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76f8d66774-gl6nr" event={"ID":"6245edcf-e1f3-4dfb-84ef-a2911f854ae4","Type":"ContainerStarted","Data":"59c760dc278a26051ad0df783029b3ac296c09349cecd2539867ccf6df1fc80a"}
Nov 04 02:06:24.370731 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:24.370708    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-76f8d66774-gl6nr"
Nov 04 02:06:24.373882 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:24.373459    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4v7xm" event={"ID":"afe4b3d8-a18c-4587-9057-83123ffe3347","Type":"ContainerStarted","Data":"ca414df06c55916aa7f14613169d3ef5c70b2298772b39e9a66bec37e75266bf"}
Nov 04 02:06:24.373882 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:24.373507    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4v7xm" event={"ID":"afe4b3d8-a18c-4587-9057-83123ffe3347","Type":"ContainerStarted","Data":"8489a036b02a437792bec4b4b87868cf1d22533417e0dcce8063835fb3508c78"}
Nov 04 02:06:24.501194 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:24.491477    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-76f8d66774-gl6nr" podStartSLOduration=2.491458614 podStartE2EDuration="2.491458614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:06:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:24.490594892 +0000 UTC m=+20.416817530" watchObservedRunningTime="2026-11-04 02:06:24.491458614 +0000 UTC m=+20.417681248"
Nov 04 02:06:24.530534 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:24.530504    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:24.708333 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:24.708244    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:24.762055 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:24.762019    4969 patch_prober.go:29] interesting pod/image-registry-6bc84fd8c9-wbbgn container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Nov 04 02:06:24.762240 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:24.762077    4969 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn" podUID="12e9dc19-8f06-4b5d-8237-3ed5a449e09f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 02:06:25.381799 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:25.381737    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-xw9tf" event={"ID":"48814638-7f4b-4276-9d0d-97b0d80f46cd","Type":"ContainerStarted","Data":"04624ec68bbc4e7abdd43f39c87710e44c80e983d63dd06b525beca98ca14fe5"}
Nov 04 02:06:25.382353 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:25.382017    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-xw9tf"
Nov 04 02:06:25.385375 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:25.385350    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8sr5s" event={"ID":"63254ae3-0393-4094-9401-e18067d30080","Type":"ContainerStarted","Data":"8d2d654f27f67be6c975c9b890f18cdeeea3aad19b124d145956c58729a9ea95"}
Nov 04 02:06:25.385488 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:25.385384    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8sr5s" event={"ID":"63254ae3-0393-4094-9401-e18067d30080","Type":"ContainerStarted","Data":"f2acbbf2f002b8b902a4db6d8d5dc1537c3831d4c70b251261f7944d637d7f62"}
Nov 04 02:06:25.385488 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:25.385410    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:06:25.388536 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:25.388496    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-xw9tf"
Nov 04 02:06:25.497303 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:25.497241    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-xw9tf" podStartSLOduration=1.70159556 podStartE2EDuration="3.497222696s" totalImagesPullingTime="1.795627136s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:06:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:25.496656459 +0000 UTC m=+21.422879101" watchObservedRunningTime="2026-11-04 02:06:25.497222696 +0000 UTC m=+21.423445333"
Nov 04 02:06:26.394950 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:26.394913    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hh6m5" event={"ID":"c41ceadb-e895-4b54-b3ea-88e0b6aa56d9","Type":"ContainerStarted","Data":"5f3603980e9413b5c954f0e98670c685efea0982186612a93e872837ada615e1"}
Nov 04 02:06:26.525009 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:26.524982    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:26.694472 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:26.694399    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:26.872726 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:26.872694    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:26.909494 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:26.909465    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-kjgvd"]
Nov 04 02:06:26.914528 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:26.914506    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-kjgvd"
Nov 04 02:06:26.966855 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:26.966790    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Nov 04 02:06:26.980215 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:26.980186    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1944cee5-2a0d-4978-badf-0471ae90f700-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-kjgvd\" (UID: \"1944cee5-2a0d-4978-badf-0471ae90f700\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kjgvd"
Nov 04 02:06:26.980348 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:26.980306    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbjkv\" (UniqueName: \"kubernetes.io/projected/1944cee5-2a0d-4978-badf-0471ae90f700-kube-api-access-tbjkv\") pod \"prometheus-operator-9cd89b74c-kjgvd\" (UID: \"1944cee5-2a0d-4978-badf-0471ae90f700\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kjgvd"
Nov 04 02:06:26.980414 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:26.980383    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1944cee5-2a0d-4978-badf-0471ae90f700-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-kjgvd\" (UID: \"1944cee5-2a0d-4978-badf-0471ae90f700\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kjgvd"
Nov 04 02:06:26.980474 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:26.980434    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1944cee5-2a0d-4978-badf-0471ae90f700-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-kjgvd\" (UID: \"1944cee5-2a0d-4978-badf-0471ae90f700\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kjgvd"
Nov 04 02:06:27.016217 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:27.016196    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Nov 04 02:06:27.016351 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:27.016200    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 04 02:06:27.016351 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:27.016243    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-7k96f\""
Nov 04 02:06:27.017563 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:27.017539    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-kjgvd"]
Nov 04 02:06:27.066220 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:27.066195    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:27.081276 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:27.081252    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1944cee5-2a0d-4978-badf-0471ae90f700-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-kjgvd\" (UID: \"1944cee5-2a0d-4978-badf-0471ae90f700\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kjgvd"
Nov 04 02:06:27.081368 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:27.081298    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbjkv\" (UniqueName: \"kubernetes.io/projected/1944cee5-2a0d-4978-badf-0471ae90f700-kube-api-access-tbjkv\") pod \"prometheus-operator-9cd89b74c-kjgvd\" (UID: \"1944cee5-2a0d-4978-badf-0471ae90f700\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kjgvd"
Nov 04 02:06:27.081368 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:27.081341    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1944cee5-2a0d-4978-badf-0471ae90f700-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-kjgvd\" (UID: \"1944cee5-2a0d-4978-badf-0471ae90f700\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kjgvd"
Nov 04 02:06:27.081478 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:27.081384    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1944cee5-2a0d-4978-badf-0471ae90f700-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-kjgvd\" (UID: \"1944cee5-2a0d-4978-badf-0471ae90f700\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kjgvd"
Nov 04 02:06:27.082110 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:27.082070    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1944cee5-2a0d-4978-badf-0471ae90f700-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-kjgvd\" (UID: \"1944cee5-2a0d-4978-badf-0471ae90f700\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kjgvd"
Nov 04 02:06:27.083828 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:27.083809    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1944cee5-2a0d-4978-badf-0471ae90f700-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-kjgvd\" (UID: \"1944cee5-2a0d-4978-badf-0471ae90f700\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kjgvd"
Nov 04 02:06:27.083997 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:27.083978    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1944cee5-2a0d-4978-badf-0471ae90f700-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-kjgvd\" (UID: \"1944cee5-2a0d-4978-badf-0471ae90f700\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kjgvd"
Nov 04 02:06:27.142349 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:27.142325    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbjkv\" (UniqueName: \"kubernetes.io/projected/1944cee5-2a0d-4978-badf-0471ae90f700-kube-api-access-tbjkv\") pod \"prometheus-operator-9cd89b74c-kjgvd\" (UID: \"1944cee5-2a0d-4978-badf-0471ae90f700\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kjgvd"
Nov 04 02:06:27.230748 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:27.230690    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-kjgvd"
Nov 04 02:06:27.608435 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:27.608401    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-kjgvd"]
Nov 04 02:06:27.641717 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:27.641691    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1944cee5_2a0d_4978_badf_0471ae90f700.slice/crio-3602dbc1df693b2ed0f4f57ec41b3456538a3ed0892e86ca57d9d779cc104dd0 WatchSource:0}: Error finding container 3602dbc1df693b2ed0f4f57ec41b3456538a3ed0892e86ca57d9d779cc104dd0: Status 404 returned error can't find the container with id 3602dbc1df693b2ed0f4f57ec41b3456538a3ed0892e86ca57d9d779cc104dd0
Nov 04 02:06:28.406005 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:28.405967    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-kjgvd" event={"ID":"1944cee5-2a0d-4978-badf-0471ae90f700","Type":"ContainerStarted","Data":"3602dbc1df693b2ed0f4f57ec41b3456538a3ed0892e86ca57d9d779cc104dd0"}
Nov 04 02:06:30.417112 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:30.417006    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-kjgvd" event={"ID":"1944cee5-2a0d-4978-badf-0471ae90f700","Type":"ContainerStarted","Data":"faa280e432dff541fedb981fe99d15aa0675d305c0e6e24b43374f8f8796f27c"}
Nov 04 02:06:30.417112 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:30.417081    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-kjgvd" event={"ID":"1944cee5-2a0d-4978-badf-0471ae90f700","Type":"ContainerStarted","Data":"e0d3ee036508b9170dc397980a2dac3fd255a78c6749abdc55c16e76c4973b34"}
Nov 04 02:06:30.535581 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:30.535524    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-kjgvd" podStartSLOduration=2.160608125 podStartE2EDuration="4.535507501s" totalImagesPullingTime="2.374899376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:06:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:30.533329119 +0000 UTC m=+26.459551763" watchObservedRunningTime="2026-11-04 02:06:30.535507501 +0000 UTC m=+26.461730134"
Nov 04 02:06:31.299849 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:31.299810    4969 patch_prober.go:29] interesting pod/image-registry-6bc84fd8c9-wbbgn container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Nov 04 02:06:31.300016 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:31.299883    4969 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn" podUID="12e9dc19-8f06-4b5d-8237-3ed5a449e09f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 02:06:32.707318 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:32.707280    4969 patch_prober.go:29] interesting pod/image-registry-5bd9d74c6b-thtzz container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Nov 04 02:06:32.707832 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:32.707349    4969 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz" podUID="cab3ee15-5d68-4840-83f4-ea8e2308e530" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 02:06:32.764712 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:32.764673    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:32.941119 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:32.938248    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:33.264572 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.264531    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:33.447922 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.447876    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:33.602816 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.602730    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-tdjrt"]
Nov 04 02:06:33.627786 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.627743    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:33.640356 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.640331    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:33.744427 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.744382    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 04 02:06:33.744835 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.744473    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 04 02:06:33.744835 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.744609    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 04 02:06:33.744835 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.744615    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xlwps\""
Nov 04 02:06:33.747034 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.747011    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/067adedf-2b86-48dd-9005-47e0178b9824-node-exporter-wtmp\") pod \"node-exporter-tdjrt\" (UID: \"067adedf-2b86-48dd-9005-47e0178b9824\") " pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:33.747240 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.747048    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/067adedf-2b86-48dd-9005-47e0178b9824-node-exporter-tls\") pod \"node-exporter-tdjrt\" (UID: \"067adedf-2b86-48dd-9005-47e0178b9824\") " pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:33.747240 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.747073    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpsdh\" (UniqueName: \"kubernetes.io/projected/067adedf-2b86-48dd-9005-47e0178b9824-kube-api-access-bpsdh\") pod \"node-exporter-tdjrt\" (UID: \"067adedf-2b86-48dd-9005-47e0178b9824\") " pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:33.747240 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.747119    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/067adedf-2b86-48dd-9005-47e0178b9824-metrics-client-ca\") pod \"node-exporter-tdjrt\" (UID: \"067adedf-2b86-48dd-9005-47e0178b9824\") " pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:33.747454 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.747245    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/067adedf-2b86-48dd-9005-47e0178b9824-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tdjrt\" (UID: \"067adedf-2b86-48dd-9005-47e0178b9824\") " pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:33.747454 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.747273    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/067adedf-2b86-48dd-9005-47e0178b9824-node-exporter-textfile\") pod \"node-exporter-tdjrt\" (UID: \"067adedf-2b86-48dd-9005-47e0178b9824\") " pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:33.747454 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.747361    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/067adedf-2b86-48dd-9005-47e0178b9824-sys\") pod \"node-exporter-tdjrt\" (UID: \"067adedf-2b86-48dd-9005-47e0178b9824\") " pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:33.747570 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.747465    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/067adedf-2b86-48dd-9005-47e0178b9824-node-exporter-accelerators-collector-config\") pod \"node-exporter-tdjrt\" (UID: \"067adedf-2b86-48dd-9005-47e0178b9824\") " pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:33.747570 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.747512    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/067adedf-2b86-48dd-9005-47e0178b9824-root\") pod \"node-exporter-tdjrt\" (UID: \"067adedf-2b86-48dd-9005-47e0178b9824\") " pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:33.817371 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.817344    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:33.848475 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.848444    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/067adedf-2b86-48dd-9005-47e0178b9824-node-exporter-accelerators-collector-config\") pod \"node-exporter-tdjrt\" (UID: \"067adedf-2b86-48dd-9005-47e0178b9824\") " pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:33.848608 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.848499    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/067adedf-2b86-48dd-9005-47e0178b9824-root\") pod \"node-exporter-tdjrt\" (UID: \"067adedf-2b86-48dd-9005-47e0178b9824\") " pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:33.848608 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.848530    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/067adedf-2b86-48dd-9005-47e0178b9824-node-exporter-wtmp\") pod \"node-exporter-tdjrt\" (UID: \"067adedf-2b86-48dd-9005-47e0178b9824\") " pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:33.848608 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.848562    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/067adedf-2b86-48dd-9005-47e0178b9824-node-exporter-tls\") pod \"node-exporter-tdjrt\" (UID: \"067adedf-2b86-48dd-9005-47e0178b9824\") " pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:33.848608 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.848583    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpsdh\" (UniqueName: \"kubernetes.io/projected/067adedf-2b86-48dd-9005-47e0178b9824-kube-api-access-bpsdh\") pod \"node-exporter-tdjrt\" (UID: \"067adedf-2b86-48dd-9005-47e0178b9824\") " pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:33.848819 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.848618    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/067adedf-2b86-48dd-9005-47e0178b9824-metrics-client-ca\") pod \"node-exporter-tdjrt\" (UID: \"067adedf-2b86-48dd-9005-47e0178b9824\") " pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:33.848819 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.848662    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/067adedf-2b86-48dd-9005-47e0178b9824-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tdjrt\" (UID: \"067adedf-2b86-48dd-9005-47e0178b9824\") " pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:33.848819 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.848688    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/067adedf-2b86-48dd-9005-47e0178b9824-node-exporter-textfile\") pod \"node-exporter-tdjrt\" (UID: \"067adedf-2b86-48dd-9005-47e0178b9824\") " pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:33.848819 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.848686    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/067adedf-2b86-48dd-9005-47e0178b9824-root\") pod \"node-exporter-tdjrt\" (UID: \"067adedf-2b86-48dd-9005-47e0178b9824\") " pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:33.848819 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.848704    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/067adedf-2b86-48dd-9005-47e0178b9824-node-exporter-wtmp\") pod \"node-exporter-tdjrt\" (UID: \"067adedf-2b86-48dd-9005-47e0178b9824\") " pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:33.848819 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.848724    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/067adedf-2b86-48dd-9005-47e0178b9824-sys\") pod \"node-exporter-tdjrt\" (UID: \"067adedf-2b86-48dd-9005-47e0178b9824\") " pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:33.848819 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.848811    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/067adedf-2b86-48dd-9005-47e0178b9824-sys\") pod \"node-exporter-tdjrt\" (UID: \"067adedf-2b86-48dd-9005-47e0178b9824\") " pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:33.849155 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.849017    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/067adedf-2b86-48dd-9005-47e0178b9824-node-exporter-textfile\") pod \"node-exporter-tdjrt\" (UID: \"067adedf-2b86-48dd-9005-47e0178b9824\") " pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:33.849155 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.849139    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/067adedf-2b86-48dd-9005-47e0178b9824-node-exporter-accelerators-collector-config\") pod \"node-exporter-tdjrt\" (UID: \"067adedf-2b86-48dd-9005-47e0178b9824\") " pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:33.849254 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.849221    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/067adedf-2b86-48dd-9005-47e0178b9824-metrics-client-ca\") pod \"node-exporter-tdjrt\" (UID: \"067adedf-2b86-48dd-9005-47e0178b9824\") " pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:33.850935 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.850910    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/067adedf-2b86-48dd-9005-47e0178b9824-node-exporter-tls\") pod \"node-exporter-tdjrt\" (UID: \"067adedf-2b86-48dd-9005-47e0178b9824\") " pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:33.851014 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.850990    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/067adedf-2b86-48dd-9005-47e0178b9824-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tdjrt\" (UID: \"067adedf-2b86-48dd-9005-47e0178b9824\") " pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:33.909282 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.909195    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpsdh\" (UniqueName: \"kubernetes.io/projected/067adedf-2b86-48dd-9005-47e0178b9824-kube-api-access-bpsdh\") pod \"node-exporter-tdjrt\" (UID: \"067adedf-2b86-48dd-9005-47e0178b9824\") " pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:33.999083 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:33.999049    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:34.013006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:34.012983    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tdjrt"
Nov 04 02:06:34.043824 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:06:34.043792    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod067adedf_2b86_48dd_9005_47e0178b9824.slice/crio-226e2aa5470bd7bb0a158ca237df4c3a45075930e717dec23cc15d087172bd3c WatchSource:0}: Error finding container 226e2aa5470bd7bb0a158ca237df4c3a45075930e717dec23cc15d087172bd3c: Status 404 returned error can't find the container with id 226e2aa5470bd7bb0a158ca237df4c3a45075930e717dec23cc15d087172bd3c
Nov 04 02:06:34.209772 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:34.209698    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:34.260836 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:34.260802    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:06:34.436360 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:34.436321    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tdjrt" event={"ID":"067adedf-2b86-48dd-9005-47e0178b9824","Type":"ContainerStarted","Data":"226e2aa5470bd7bb0a158ca237df4c3a45075930e717dec23cc15d087172bd3c"}
Nov 04 02:06:34.759441 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:34.759397    4969 patch_prober.go:29] interesting pod/image-registry-6bc84fd8c9-wbbgn container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Nov 04 02:06:34.759873 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:34.759460    4969 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn" podUID="12e9dc19-8f06-4b5d-8237-3ed5a449e09f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 02:06:34.988036 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:34.988005    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:35.177744 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:35.177657    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:35.398158 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:35.398132    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8sr5s"
Nov 04 02:06:35.867840 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:35.867809    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:37.452167 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:37.452128    4969 generic.go:356] "Generic (PLEG): container finished" podID="067adedf-2b86-48dd-9005-47e0178b9824" containerID="786738412da84f339ae8979ae2fd625d88bb1f3119f6145ccaa6f898ae704e78" exitCode=0
Nov 04 02:06:37.452642 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:37.452198    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tdjrt" event={"ID":"067adedf-2b86-48dd-9005-47e0178b9824","Type":"ContainerDied","Data":"786738412da84f339ae8979ae2fd625d88bb1f3119f6145ccaa6f898ae704e78"}
Nov 04 02:06:37.727132 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:37.727040    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:37.902849 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:37.902812    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:38.105517 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:38.105482    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:38.258793 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:38.258761    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-98h6z"
Nov 04 02:06:38.287502 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:38.287467    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:38.459035 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:38.458952    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tdjrt" event={"ID":"067adedf-2b86-48dd-9005-47e0178b9824","Type":"ContainerStarted","Data":"7ba99050951f36dbc71b743a76777703e526390036a53a709166175ea82bb16c"}
Nov 04 02:06:38.459035 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:38.459009    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tdjrt" event={"ID":"067adedf-2b86-48dd-9005-47e0178b9824","Type":"ContainerStarted","Data":"23606f115dc3c8416632f96950c300abca7a29f55ccdb47734241cfbf4e779f0"}
Nov 04 02:06:38.476932 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:38.476905    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:38.578741 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:38.578684    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-tdjrt" podStartSLOduration=3.130504925 podStartE2EDuration="5.578668698s" totalImagesPullingTime="2.361509489s" totalInitContainerRuntime="86.654284ms" isStatefulPod=true podCreationTimestamp="2026-11-04 02:06:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:38.57712931 +0000 UTC m=+34.503351943" watchObservedRunningTime="2026-11-04 02:06:38.578668698 +0000 UTC m=+34.504891331"
Nov 04 02:06:38.657047 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:38.657017    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:39.069616 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:39.069445    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:39.253108 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:39.253061    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:39.525868 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:39.525838    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:39.702366 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:39.702328    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:40.393822 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:40.393780    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:40.575929 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:40.575896    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:40.752273 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:40.752191    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:41.299750 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:41.299722    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:06:41.962864 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:41.962759    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:42.135626 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:42.135564    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:42.480813 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:42.480772    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-2qtvl" event={"ID":"98621f1f-0f97-44be-995b-9a37a0217747","Type":"ContainerStarted","Data":"8ff09f42da8acdc8b65b2420b8c8b74f702a47d7151d4316deb90fa981efe885"}
Nov 04 02:06:42.481009 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:42.480944    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-2qtvl"
Nov 04 02:06:42.497751 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:42.497721    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-2qtvl"
Nov 04 02:06:42.660137 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:42.660071    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:42.705894 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:42.705809    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/downloads-6dd789c8b7-2qtvl" podStartSLOduration=2.689580265 podStartE2EDuration="20.705789708s" totalImagesPullingTime="18.016209443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:06:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:06:42.655451991 +0000 UTC m=+38.581674624" watchObservedRunningTime="2026-11-04 02:06:42.705789708 +0000 UTC m=+38.632012341"
Nov 04 02:06:42.709234 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:42.709209    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:06:42.856830 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:42.856796    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:43.134429 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:43.134346    4969 patch_prober.go:29] interesting pod/image-registry-76f8d66774-gl6nr container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Nov 04 02:06:43.134429 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:43.134403    4969 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-76f8d66774-gl6nr" podUID="6245edcf-e1f3-4dfb-84ef-a2911f854ae4" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 02:06:44.692183 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:44.692150    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:44.892329 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:44.892298    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:45.223056 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:45.223015    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:45.409665 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:45.409615    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:46.400248 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:46.400220    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-76f8d66774-gl6nr"
Nov 04 02:06:46.626714 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:46.626670    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"]
Nov 04 02:06:46.751527 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:46.751454    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:46.920755 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:46.920720    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:47.101056 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:47.101019    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:47.726998 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:47.726956    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz" podUID="cab3ee15-5d68-4840-83f4-ea8e2308e530" containerName="registry" containerID="cri-o://5ea586b54622e0bfec7ef91bec4922b7f90488d84f50b5638568da3f04a402fe" gracePeriod=30
Nov 04 02:06:48.026654 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.026629    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:06:48.068555 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.068529    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cab3ee15-5d68-4840-83f4-ea8e2308e530-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cab3ee15-5d68-4840-83f4-ea8e2308e530-installation-pull-secrets\") pod \"cab3ee15-5d68-4840-83f4-ea8e2308e530\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") "
Nov 04 02:06:48.068690 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.068572    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cab3ee15-5d68-4840-83f4-ea8e2308e530-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cab3ee15-5d68-4840-83f4-ea8e2308e530-image-registry-private-configuration\") pod \"cab3ee15-5d68-4840-83f4-ea8e2308e530\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") "
Nov 04 02:06:48.068690 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.068605    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cab3ee15-5d68-4840-83f4-ea8e2308e530-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cab3ee15-5d68-4840-83f4-ea8e2308e530-trusted-ca\") pod \"cab3ee15-5d68-4840-83f4-ea8e2308e530\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") "
Nov 04 02:06:48.068690 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.068637    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-certificates\") pod \"cab3ee15-5d68-4840-83f4-ea8e2308e530\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") "
Nov 04 02:06:48.068853 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.068691    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cab3ee15-5d68-4840-83f4-ea8e2308e530-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cab3ee15-5d68-4840-83f4-ea8e2308e530-ca-trust-extracted\") pod \"cab3ee15-5d68-4840-83f4-ea8e2308e530\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") "
Nov 04 02:06:48.068853 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.068741    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-bound-sa-token\") pod \"cab3ee15-5d68-4840-83f4-ea8e2308e530\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") "
Nov 04 02:06:48.068853 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.068767    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls\" (UniqueName: \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls\") pod \"cab3ee15-5d68-4840-83f4-ea8e2308e530\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") "
Nov 04 02:06:48.068853 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.068821    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-kube-api-access-h77nn\" (UniqueName: \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-kube-api-access-h77nn\") pod \"cab3ee15-5d68-4840-83f4-ea8e2308e530\" (UID: \"cab3ee15-5d68-4840-83f4-ea8e2308e530\") "
Nov 04 02:06:48.069055 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.068965    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cab3ee15-5d68-4840-83f4-ea8e2308e530-trusted-ca" pod "cab3ee15-5d68-4840-83f4-ea8e2308e530" (UID: "cab3ee15-5d68-4840-83f4-ea8e2308e530"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:06:48.069130 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.069070    4969 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cab3ee15-5d68-4840-83f4-ea8e2308e530-trusted-ca\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:06:48.069130 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.069074    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-certificates" pod "cab3ee15-5d68-4840-83f4-ea8e2308e530" (UID: "cab3ee15-5d68-4840-83f4-ea8e2308e530"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:06:48.071627 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.071554    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cab3ee15-5d68-4840-83f4-ea8e2308e530-installation-pull-secrets" pod "cab3ee15-5d68-4840-83f4-ea8e2308e530" (UID: "cab3ee15-5d68-4840-83f4-ea8e2308e530"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:06:48.072011 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.071973    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cab3ee15-5d68-4840-83f4-ea8e2308e530-image-registry-private-configuration" pod "cab3ee15-5d68-4840-83f4-ea8e2308e530" (UID: "cab3ee15-5d68-4840-83f4-ea8e2308e530"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:06:48.072216 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.072164    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-bound-sa-token" pod "cab3ee15-5d68-4840-83f4-ea8e2308e530" (UID: "cab3ee15-5d68-4840-83f4-ea8e2308e530"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:06:48.072303 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.072280    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-kube-api-access-h77nn" pod "cab3ee15-5d68-4840-83f4-ea8e2308e530" (UID: "cab3ee15-5d68-4840-83f4-ea8e2308e530"). InnerVolumeSpecName "kube-api-access-h77nn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:06:48.072359 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.072334    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls" pod "cab3ee15-5d68-4840-83f4-ea8e2308e530" (UID: "cab3ee15-5d68-4840-83f4-ea8e2308e530"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:06:48.078280 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.078259    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cab3ee15-5d68-4840-83f4-ea8e2308e530-ca-trust-extracted" pod "cab3ee15-5d68-4840-83f4-ea8e2308e530" (UID: "cab3ee15-5d68-4840-83f4-ea8e2308e530"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:06:48.169744 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.169712    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h77nn\" (UniqueName: \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-kube-api-access-h77nn\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:06:48.169881 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.169747    4969 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cab3ee15-5d68-4840-83f4-ea8e2308e530-installation-pull-secrets\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:06:48.169881 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.169775    4969 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cab3ee15-5d68-4840-83f4-ea8e2308e530-image-registry-private-configuration\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:06:48.169881 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.169791    4969 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-certificates\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:06:48.169881 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.169808    4969 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cab3ee15-5d68-4840-83f4-ea8e2308e530-ca-trust-extracted\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:06:48.169881 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.169822    4969 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-bound-sa-token\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:06:48.169881 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.169836    4969 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cab3ee15-5d68-4840-83f4-ea8e2308e530-registry-tls\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:06:48.513633 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.513595    4969 generic.go:356] "Generic (PLEG): container finished" podID="cab3ee15-5d68-4840-83f4-ea8e2308e530" containerID="5ea586b54622e0bfec7ef91bec4922b7f90488d84f50b5638568da3f04a402fe" exitCode=0
Nov 04 02:06:48.513819 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.513656    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz"
Nov 04 02:06:48.513819 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.513663    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz" event={"ID":"cab3ee15-5d68-4840-83f4-ea8e2308e530","Type":"ContainerDied","Data":"5ea586b54622e0bfec7ef91bec4922b7f90488d84f50b5638568da3f04a402fe"}
Nov 04 02:06:48.513819 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.513807    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bd9d74c6b-thtzz" event={"ID":"cab3ee15-5d68-4840-83f4-ea8e2308e530","Type":"ContainerDied","Data":"f183641efc5ebde82c8a28e5630870e3933a7fb21a6bb12f5fa3089bfcb85b1e"}
Nov 04 02:06:48.513974 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.513845    4969 scope.go:122] "RemoveContainer" containerID="5ea586b54622e0bfec7ef91bec4922b7f90488d84f50b5638568da3f04a402fe"
Nov 04 02:06:48.525295 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.525276    4969 scope.go:122] "RemoveContainer" containerID="5ea586b54622e0bfec7ef91bec4922b7f90488d84f50b5638568da3f04a402fe"
Nov 04 02:06:48.525717 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:06:48.525693    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ea586b54622e0bfec7ef91bec4922b7f90488d84f50b5638568da3f04a402fe\": container with ID starting with 5ea586b54622e0bfec7ef91bec4922b7f90488d84f50b5638568da3f04a402fe not found: ID does not exist" containerID="5ea586b54622e0bfec7ef91bec4922b7f90488d84f50b5638568da3f04a402fe"
Nov 04 02:06:48.525806 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.525729    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ea586b54622e0bfec7ef91bec4922b7f90488d84f50b5638568da3f04a402fe"} err="failed to get container status \"5ea586b54622e0bfec7ef91bec4922b7f90488d84f50b5638568da3f04a402fe\": rpc error: code = NotFound desc = could not find container \"5ea586b54622e0bfec7ef91bec4922b7f90488d84f50b5638568da3f04a402fe\": container with ID starting with 5ea586b54622e0bfec7ef91bec4922b7f90488d84f50b5638568da3f04a402fe not found: ID does not exist"
Nov 04 02:06:48.676672 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.676637    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5bd9d74c6b-thtzz"]
Nov 04 02:06:48.726056 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:48.726026    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5bd9d74c6b-thtzz"]
Nov 04 02:06:50.200230 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:50.200198    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cab3ee15-5d68-4840-83f4-ea8e2308e530" path="/var/lib/kubelet/pods/cab3ee15-5d68-4840-83f4-ea8e2308e530/volumes"
Nov 04 02:06:52.255645 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:52.255610    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:52.459455 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:52.459426    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:54.035954 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:54.035921    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:54.247663 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:54.247635    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:54.544265 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:54.544236    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:54.752429 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:54.752402    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:55.881436 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:55.881405    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:56.062700 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:56.062671    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:56.887321 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:56.887284    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:57.070637 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:57.070603    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:59.426885 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:59.426857    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:06:59.641082 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:06:59.641053    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:04.261180 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:07:04.261143    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:07:05.117797 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:05.117760    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:05.294895 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:05.294859    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:06.011265 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:06.011234    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:06.192688 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:06.192658    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:09.527543 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:09.527507    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:09.727525 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:09.727495    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:11.651221 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:11.651162    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn" podUID="12e9dc19-8f06-4b5d-8237-3ed5a449e09f" containerName="registry" containerID="cri-o://3301eed52149903e3de804984c04647f3a629fd88b0e0e99f5851b47131edd01" gracePeriod=30
Nov 04 02:07:11.922269 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:11.922215    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:07:11.935421 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:11.935399    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-bound-sa-token\") pod \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") "
Nov 04 02:07:11.935527 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:11.935456    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-ca-trust-extracted\") pod \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") "
Nov 04 02:07:11.935527 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:11.935488    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls\" (UniqueName: \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls\") pod \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") "
Nov 04 02:07:11.935601 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:11.935536    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-trusted-ca\") pod \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") "
Nov 04 02:07:11.935681 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:11.935659    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-installation-pull-secrets\") pod \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") "
Nov 04 02:07:11.935742 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:11.935712    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-certificates\") pod \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") "
Nov 04 02:07:11.935801 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:11.935778    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-image-registry-private-configuration\") pod \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") "
Nov 04 02:07:11.935874 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:11.935858    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-kube-api-access-l4gp2\" (UniqueName: \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-kube-api-access-l4gp2\") pod \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\" (UID: \"12e9dc19-8f06-4b5d-8237-3ed5a449e09f\") "
Nov 04 02:07:11.936006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:11.935983    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-trusted-ca" pod "12e9dc19-8f06-4b5d-8237-3ed5a449e09f" (UID: "12e9dc19-8f06-4b5d-8237-3ed5a449e09f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:07:11.936087 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:11.936056    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-certificates" pod "12e9dc19-8f06-4b5d-8237-3ed5a449e09f" (UID: "12e9dc19-8f06-4b5d-8237-3ed5a449e09f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:07:11.936262 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:11.936200    4969 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-trusted-ca\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:07:11.936262 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:11.936223    4969 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-certificates\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:07:11.937792 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:11.937757    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-bound-sa-token" pod "12e9dc19-8f06-4b5d-8237-3ed5a449e09f" (UID: "12e9dc19-8f06-4b5d-8237-3ed5a449e09f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:07:11.937894 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:11.937847    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-installation-pull-secrets" pod "12e9dc19-8f06-4b5d-8237-3ed5a449e09f" (UID: "12e9dc19-8f06-4b5d-8237-3ed5a449e09f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:07:11.938251 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:11.938229    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls" pod "12e9dc19-8f06-4b5d-8237-3ed5a449e09f" (UID: "12e9dc19-8f06-4b5d-8237-3ed5a449e09f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:07:11.938504 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:11.938475    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-image-registry-private-configuration" pod "12e9dc19-8f06-4b5d-8237-3ed5a449e09f" (UID: "12e9dc19-8f06-4b5d-8237-3ed5a449e09f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:07:11.938504 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:11.938497    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-kube-api-access-l4gp2" pod "12e9dc19-8f06-4b5d-8237-3ed5a449e09f" (UID: "12e9dc19-8f06-4b5d-8237-3ed5a449e09f"). InnerVolumeSpecName "kube-api-access-l4gp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:07:11.942821 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:11.942798    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-ca-trust-extracted" pod "12e9dc19-8f06-4b5d-8237-3ed5a449e09f" (UID: "12e9dc19-8f06-4b5d-8237-3ed5a449e09f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:07:12.036966 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:12.036942    4969 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-installation-pull-secrets\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:07:12.037069 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:12.036968    4969 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-image-registry-private-configuration\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:07:12.037069 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:12.036983    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4gp2\" (UniqueName: \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-kube-api-access-l4gp2\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:07:12.037069 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:12.036997    4969 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-bound-sa-token\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:07:12.037069 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:12.037011    4969 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-ca-trust-extracted\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:07:12.037069 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:12.037023    4969 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12e9dc19-8f06-4b5d-8237-3ed5a449e09f-registry-tls\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:07:12.413532 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:12.413503    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:12.587580 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:12.587549    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:12.618782 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:12.618756    4969 generic.go:356] "Generic (PLEG): container finished" podID="12e9dc19-8f06-4b5d-8237-3ed5a449e09f" containerID="3301eed52149903e3de804984c04647f3a629fd88b0e0e99f5851b47131edd01" exitCode=0
Nov 04 02:07:12.618900 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:12.618801    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"
Nov 04 02:07:12.618900 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:12.618840    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn" event={"ID":"12e9dc19-8f06-4b5d-8237-3ed5a449e09f","Type":"ContainerDied","Data":"3301eed52149903e3de804984c04647f3a629fd88b0e0e99f5851b47131edd01"}
Nov 04 02:07:12.618900 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:12.618887    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6bc84fd8c9-wbbgn" event={"ID":"12e9dc19-8f06-4b5d-8237-3ed5a449e09f","Type":"ContainerDied","Data":"41d25760db43fb80c551b6bcbd2dffc15a34cc035bb728cf3f4d5ddef0db1d2a"}
Nov 04 02:07:12.619003 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:12.618910    4969 scope.go:122] "RemoveContainer" containerID="3301eed52149903e3de804984c04647f3a629fd88b0e0e99f5851b47131edd01"
Nov 04 02:07:12.629221 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:12.629071    4969 scope.go:122] "RemoveContainer" containerID="3301eed52149903e3de804984c04647f3a629fd88b0e0e99f5851b47131edd01"
Nov 04 02:07:12.629409 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:07:12.629390    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3301eed52149903e3de804984c04647f3a629fd88b0e0e99f5851b47131edd01\": container with ID starting with 3301eed52149903e3de804984c04647f3a629fd88b0e0e99f5851b47131edd01 not found: ID does not exist" containerID="3301eed52149903e3de804984c04647f3a629fd88b0e0e99f5851b47131edd01"
Nov 04 02:07:12.629457 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:12.629418    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3301eed52149903e3de804984c04647f3a629fd88b0e0e99f5851b47131edd01"} err="failed to get container status \"3301eed52149903e3de804984c04647f3a629fd88b0e0e99f5851b47131edd01\": rpc error: code = NotFound desc = could not find container \"3301eed52149903e3de804984c04647f3a629fd88b0e0e99f5851b47131edd01\": container with ID starting with 3301eed52149903e3de804984c04647f3a629fd88b0e0e99f5851b47131edd01 not found: ID does not exist"
Nov 04 02:07:12.830579 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:12.830556    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"]
Nov 04 02:07:12.880312 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:12.880257    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6bc84fd8c9-wbbgn"]
Nov 04 02:07:14.202126 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:14.202078    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="12e9dc19-8f06-4b5d-8237-3ed5a449e09f" path="/var/lib/kubelet/pods/12e9dc19-8f06-4b5d-8237-3ed5a449e09f/volumes"
Nov 04 02:07:17.641549 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:17.641514    4969 generic.go:356] "Generic (PLEG): container finished" podID="150189f8-cd39-4a16-9d07-112edc6e83dc" containerID="6628305f8b5bf2ad7a390949b562af58cf024a97af31356d0a6cef6c460a41c4" exitCode=0
Nov 04 02:07:17.641944 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:17.641587    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-c2vfk" event={"ID":"150189f8-cd39-4a16-9d07-112edc6e83dc","Type":"ContainerDied","Data":"6628305f8b5bf2ad7a390949b562af58cf024a97af31356d0a6cef6c460a41c4"}
Nov 04 02:07:17.641944 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:17.641908    4969 scope.go:122] "RemoveContainer" containerID="6628305f8b5bf2ad7a390949b562af58cf024a97af31356d0a6cef6c460a41c4"
Nov 04 02:07:18.648779 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:18.648748    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-c2vfk" event={"ID":"150189f8-cd39-4a16-9d07-112edc6e83dc","Type":"ContainerStarted","Data":"05e90d698477f96ce09ae36a774570c76eb6731478acaa7b08edbc4610665197"}
Nov 04 02:07:19.655328 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:19.655299    4969 generic.go:356] "Generic (PLEG): container finished" podID="a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d" containerID="006534a5f46ce8db8e724bbecbf66bdf13f290d4edd54f696aa73d49be3d399e" exitCode=0
Nov 04 02:07:19.655656 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:19.655338    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-96g6r" event={"ID":"a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d","Type":"ContainerDied","Data":"006534a5f46ce8db8e724bbecbf66bdf13f290d4edd54f696aa73d49be3d399e"}
Nov 04 02:07:19.655656 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:19.655621    4969 scope.go:122] "RemoveContainer" containerID="006534a5f46ce8db8e724bbecbf66bdf13f290d4edd54f696aa73d49be3d399e"
Nov 04 02:07:20.662056 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:20.662017    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-96g6r" event={"ID":"a9b6f7fe-dea9-4dd0-9a5f-7dbc9c68b54d","Type":"ContainerStarted","Data":"4fdc03891be000ed86e227e192ae095a5c12e247d17f2b106b629bae6d6354b2"}
Nov 04 02:07:21.336072 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:21.336038    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:21.504804 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:21.504775    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:22.688390 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:22.688355    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:22.867575 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:22.867545    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:24.287275 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:24.287248    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:24.463110 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:24.463048    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:24.641269 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:24.641202    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:25.682664 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:25.682636    4969 generic.go:356] "Generic (PLEG): container finished" podID="7cdf1fc0-00ea-4108-8ff0-5c031a193968" containerID="5116f217e0071fb53e23496020a2322a85e2ca08f45dc488b3d0f5c0daa79e7d" exitCode=0
Nov 04 02:07:25.683008 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:25.682718    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-6cc2t" event={"ID":"7cdf1fc0-00ea-4108-8ff0-5c031a193968","Type":"ContainerDied","Data":"5116f217e0071fb53e23496020a2322a85e2ca08f45dc488b3d0f5c0daa79e7d"}
Nov 04 02:07:25.683113 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:25.683075    4969 scope.go:122] "RemoveContainer" containerID="5116f217e0071fb53e23496020a2322a85e2ca08f45dc488b3d0f5c0daa79e7d"
Nov 04 02:07:26.087288 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:26.087256    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:26.691779 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:26.691734    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-6cc2t" event={"ID":"7cdf1fc0-00ea-4108-8ff0-5c031a193968","Type":"ContainerStarted","Data":"18c7a83ed2a398c7e1bf01b8cf30eed166487b9813c59cca932c293b86f54ea4"}
Nov 04 02:07:26.707382 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:26.707358    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-xwzqv-7rqm2"]
Nov 04 02:07:26.707734 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:26.707702    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-xwzqv-7rqm2" podUID="eb9e13cc-14be-46c7-bea5-2501b0a85486" containerName="insights-gathering" containerID="cri-o://594defed3b74095e62acf6c62217a28e2c22a2c6f42dba6f0f16d064830846a9" gracePeriod=30
Nov 04 02:07:27.025281 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:27.025257    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-xwzqv-7rqm2"
Nov 04 02:07:27.149166 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:27.149136    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb9e13cc-14be-46c7-bea5-2501b0a85486-kube-api-access-qm6tq\" (UniqueName: \"kubernetes.io/projected/eb9e13cc-14be-46c7-bea5-2501b0a85486-kube-api-access-qm6tq\") pod \"eb9e13cc-14be-46c7-bea5-2501b0a85486\" (UID: \"eb9e13cc-14be-46c7-bea5-2501b0a85486\") "
Nov 04 02:07:27.149303 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:27.149172    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb9e13cc-14be-46c7-bea5-2501b0a85486-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb9e13cc-14be-46c7-bea5-2501b0a85486-service-ca-bundle\") pod \"eb9e13cc-14be-46c7-bea5-2501b0a85486\" (UID: \"eb9e13cc-14be-46c7-bea5-2501b0a85486\") "
Nov 04 02:07:27.149303 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:27.149208    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb9e13cc-14be-46c7-bea5-2501b0a85486-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb9e13cc-14be-46c7-bea5-2501b0a85486-trusted-ca-bundle\") pod \"eb9e13cc-14be-46c7-bea5-2501b0a85486\" (UID: \"eb9e13cc-14be-46c7-bea5-2501b0a85486\") "
Nov 04 02:07:27.149303 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:27.149261    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb9e13cc-14be-46c7-bea5-2501b0a85486-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/eb9e13cc-14be-46c7-bea5-2501b0a85486-archives-path\") pod \"eb9e13cc-14be-46c7-bea5-2501b0a85486\" (UID: \"eb9e13cc-14be-46c7-bea5-2501b0a85486\") "
Nov 04 02:07:27.149534 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:27.149508    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb9e13cc-14be-46c7-bea5-2501b0a85486-service-ca-bundle" pod "eb9e13cc-14be-46c7-bea5-2501b0a85486" (UID: "eb9e13cc-14be-46c7-bea5-2501b0a85486"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:07:27.149629 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:27.149608    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb9e13cc-14be-46c7-bea5-2501b0a85486-trusted-ca-bundle" pod "eb9e13cc-14be-46c7-bea5-2501b0a85486" (UID: "eb9e13cc-14be-46c7-bea5-2501b0a85486"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:07:27.149698 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:27.149668    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb9e13cc-14be-46c7-bea5-2501b0a85486-archives-path" pod "eb9e13cc-14be-46c7-bea5-2501b0a85486" (UID: "eb9e13cc-14be-46c7-bea5-2501b0a85486"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:07:27.151452 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:27.151429    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb9e13cc-14be-46c7-bea5-2501b0a85486-kube-api-access-qm6tq" pod "eb9e13cc-14be-46c7-bea5-2501b0a85486" (UID: "eb9e13cc-14be-46c7-bea5-2501b0a85486"). InnerVolumeSpecName "kube-api-access-qm6tq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:07:27.250489 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:27.250439    4969 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/eb9e13cc-14be-46c7-bea5-2501b0a85486-archives-path\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:07:27.250489 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:27.250459    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qm6tq\" (UniqueName: \"kubernetes.io/projected/eb9e13cc-14be-46c7-bea5-2501b0a85486-kube-api-access-qm6tq\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:07:27.250489 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:27.250472    4969 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb9e13cc-14be-46c7-bea5-2501b0a85486-service-ca-bundle\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:07:27.250489 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:27.250481    4969 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb9e13cc-14be-46c7-bea5-2501b0a85486-trusted-ca-bundle\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:07:27.697684 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:27.697647    4969 generic.go:356] "Generic (PLEG): container finished" podID="eb9e13cc-14be-46c7-bea5-2501b0a85486" containerID="594defed3b74095e62acf6c62217a28e2c22a2c6f42dba6f0f16d064830846a9" exitCode=2
Nov 04 02:07:27.698228 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:27.697698    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-xwzqv-7rqm2"
Nov 04 02:07:27.698228 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:27.697711    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-xwzqv-7rqm2" event={"ID":"eb9e13cc-14be-46c7-bea5-2501b0a85486","Type":"ContainerDied","Data":"594defed3b74095e62acf6c62217a28e2c22a2c6f42dba6f0f16d064830846a9"}
Nov 04 02:07:27.698228 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:27.697760    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-xwzqv-7rqm2" event={"ID":"eb9e13cc-14be-46c7-bea5-2501b0a85486","Type":"ContainerDied","Data":"c6d2b1ba17da682935416cb2e38cf5c0b7197bfa360dc9c158dd133b3fc93a77"}
Nov 04 02:07:27.698228 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:27.697782    4969 scope.go:122] "RemoveContainer" containerID="594defed3b74095e62acf6c62217a28e2c22a2c6f42dba6f0f16d064830846a9"
Nov 04 02:07:27.706900 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:27.706875    4969 scope.go:122] "RemoveContainer" containerID="594defed3b74095e62acf6c62217a28e2c22a2c6f42dba6f0f16d064830846a9"
Nov 04 02:07:27.707219 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:07:27.707198    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"594defed3b74095e62acf6c62217a28e2c22a2c6f42dba6f0f16d064830846a9\": container with ID starting with 594defed3b74095e62acf6c62217a28e2c22a2c6f42dba6f0f16d064830846a9 not found: ID does not exist" containerID="594defed3b74095e62acf6c62217a28e2c22a2c6f42dba6f0f16d064830846a9"
Nov 04 02:07:27.707297 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:27.707233    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"594defed3b74095e62acf6c62217a28e2c22a2c6f42dba6f0f16d064830846a9"} err="failed to get container status \"594defed3b74095e62acf6c62217a28e2c22a2c6f42dba6f0f16d064830846a9\": rpc error: code = NotFound desc = could not find container \"594defed3b74095e62acf6c62217a28e2c22a2c6f42dba6f0f16d064830846a9\": container with ID starting with 594defed3b74095e62acf6c62217a28e2c22a2c6f42dba6f0f16d064830846a9 not found: ID does not exist"
Nov 04 02:07:27.860203 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:27.860178    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-xwzqv-7rqm2"]
Nov 04 02:07:27.909610 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:27.909583    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-xwzqv-7rqm2"]
Nov 04 02:07:28.199987 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:28.199961    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb9e13cc-14be-46c7-bea5-2501b0a85486" path="/var/lib/kubelet/pods/eb9e13cc-14be-46c7-bea5-2501b0a85486/volumes"
Nov 04 02:07:34.258956 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:07:34.258914    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:07:39.538586 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:39.538548    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:39.715963 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:39.715932    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:07:54.389483 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:07:54.389445    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:04.259814 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:08:04.259783    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:08:34.258970 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:08:34.258939    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:08:54.743925 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:08:54.743889    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:55.108409 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:08:55.108377    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:08:55.283971 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:08:55.283939    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:04.259288 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:09:04.259257    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:09:24.031057 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:24.031022    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:24.243647 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:24.243615    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:25.718111 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:25.718063    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:25.893029 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:25.893000    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:27.059892 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:27.059861    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:32.319997 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:32.319953    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:32.511369 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:32.511336    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:33.540828 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.540796    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"]
Nov 04 02:09:33.541254 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.541144    4969 cpu_manager.go:414] "containerMap: removing container" podUID="12e9dc19-8f06-4b5d-8237-3ed5a449e09f" containerName="registry"
Nov 04 02:09:33.541254 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.541157    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12e9dc19-8f06-4b5d-8237-3ed5a449e09f" containerName="registry"
Nov 04 02:09:33.541254 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.541167    4969 cpu_manager.go:414] "containerMap: removing container" podUID="eb9e13cc-14be-46c7-bea5-2501b0a85486" containerName="insights-gathering"
Nov 04 02:09:33.541254 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.541173    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb9e13cc-14be-46c7-bea5-2501b0a85486" containerName="insights-gathering"
Nov 04 02:09:33.541254 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.541182    4969 cpu_manager.go:414] "containerMap: removing container" podUID="cab3ee15-5d68-4840-83f4-ea8e2308e530" containerName="registry"
Nov 04 02:09:33.541254 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.541187    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cab3ee15-5d68-4840-83f4-ea8e2308e530" containerName="registry"
Nov 04 02:09:33.541471 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.541260    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="12e9dc19-8f06-4b5d-8237-3ed5a449e09f" containerName="registry"
Nov 04 02:09:33.541471 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.541267    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb9e13cc-14be-46c7-bea5-2501b0a85486" containerName="insights-gathering"
Nov 04 02:09:33.541471 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.541273    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="cab3ee15-5d68-4840-83f4-ea8e2308e530" containerName="registry"
Nov 04 02:09:33.543191 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.543174    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.595749 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.595726    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 04 02:09:33.621714 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.621693    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-vnp8f\""
Nov 04 02:09:33.648811 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.648789    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d4bc677-6239-47b2-92c1-1924e9cd1b54-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-8bglg\" (UID: \"6d4bc677-6239-47b2-92c1-1924e9cd1b54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.648909 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.648821    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d4bc677-6239-47b2-92c1-1924e9cd1b54-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-8bglg\" (UID: \"6d4bc677-6239-47b2-92c1-1924e9cd1b54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.648909 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.648839    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2xzm\" (UniqueName: \"kubernetes.io/projected/6d4bc677-6239-47b2-92c1-1924e9cd1b54-kube-api-access-d2xzm\") pod \"maas-default-gateway-openshift-default-6cc458d887-8bglg\" (UID: \"6d4bc677-6239-47b2-92c1-1924e9cd1b54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.648985 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.648917    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d4bc677-6239-47b2-92c1-1924e9cd1b54-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-8bglg\" (UID: \"6d4bc677-6239-47b2-92c1-1924e9cd1b54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.648985 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.648955    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d4bc677-6239-47b2-92c1-1924e9cd1b54-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-8bglg\" (UID: \"6d4bc677-6239-47b2-92c1-1924e9cd1b54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.649047 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.649003    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d4bc677-6239-47b2-92c1-1924e9cd1b54-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-8bglg\" (UID: \"6d4bc677-6239-47b2-92c1-1924e9cd1b54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.649047 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.649026    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d4bc677-6239-47b2-92c1-1924e9cd1b54-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-8bglg\" (UID: \"6d4bc677-6239-47b2-92c1-1924e9cd1b54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.649123 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.649056    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d4bc677-6239-47b2-92c1-1924e9cd1b54-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-8bglg\" (UID: \"6d4bc677-6239-47b2-92c1-1924e9cd1b54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.649123 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.649104    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d4bc677-6239-47b2-92c1-1924e9cd1b54-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-8bglg\" (UID: \"6d4bc677-6239-47b2-92c1-1924e9cd1b54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.659260 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.659239    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"]
Nov 04 02:09:33.683085 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.683065    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:33.749709 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.749684    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d4bc677-6239-47b2-92c1-1924e9cd1b54-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-8bglg\" (UID: \"6d4bc677-6239-47b2-92c1-1924e9cd1b54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.749797 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.749713    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d4bc677-6239-47b2-92c1-1924e9cd1b54-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-8bglg\" (UID: \"6d4bc677-6239-47b2-92c1-1924e9cd1b54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.749797 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.749730    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d4bc677-6239-47b2-92c1-1924e9cd1b54-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-8bglg\" (UID: \"6d4bc677-6239-47b2-92c1-1924e9cd1b54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.749797 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.749753    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d4bc677-6239-47b2-92c1-1924e9cd1b54-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-8bglg\" (UID: \"6d4bc677-6239-47b2-92c1-1924e9cd1b54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.749797 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.749791    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d4bc677-6239-47b2-92c1-1924e9cd1b54-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-8bglg\" (UID: \"6d4bc677-6239-47b2-92c1-1924e9cd1b54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.749978 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.749854    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d4bc677-6239-47b2-92c1-1924e9cd1b54-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-8bglg\" (UID: \"6d4bc677-6239-47b2-92c1-1924e9cd1b54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.749978 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.749886    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d4bc677-6239-47b2-92c1-1924e9cd1b54-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-8bglg\" (UID: \"6d4bc677-6239-47b2-92c1-1924e9cd1b54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.749978 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.749913    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2xzm\" (UniqueName: \"kubernetes.io/projected/6d4bc677-6239-47b2-92c1-1924e9cd1b54-kube-api-access-d2xzm\") pod \"maas-default-gateway-openshift-default-6cc458d887-8bglg\" (UID: \"6d4bc677-6239-47b2-92c1-1924e9cd1b54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.749978 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.749949    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d4bc677-6239-47b2-92c1-1924e9cd1b54-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-8bglg\" (UID: \"6d4bc677-6239-47b2-92c1-1924e9cd1b54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.750200 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.750169    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d4bc677-6239-47b2-92c1-1924e9cd1b54-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-8bglg\" (UID: \"6d4bc677-6239-47b2-92c1-1924e9cd1b54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.750260 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.750212    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d4bc677-6239-47b2-92c1-1924e9cd1b54-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-8bglg\" (UID: \"6d4bc677-6239-47b2-92c1-1924e9cd1b54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.750485 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.750463    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d4bc677-6239-47b2-92c1-1924e9cd1b54-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-8bglg\" (UID: \"6d4bc677-6239-47b2-92c1-1924e9cd1b54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.750570 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.750507    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d4bc677-6239-47b2-92c1-1924e9cd1b54-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-8bglg\" (UID: \"6d4bc677-6239-47b2-92c1-1924e9cd1b54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.750719 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.750698    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d4bc677-6239-47b2-92c1-1924e9cd1b54-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-8bglg\" (UID: \"6d4bc677-6239-47b2-92c1-1924e9cd1b54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.752014 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.751997    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d4bc677-6239-47b2-92c1-1924e9cd1b54-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-8bglg\" (UID: \"6d4bc677-6239-47b2-92c1-1924e9cd1b54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.752524 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.752507    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d4bc677-6239-47b2-92c1-1924e9cd1b54-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-8bglg\" (UID: \"6d4bc677-6239-47b2-92c1-1924e9cd1b54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.810058 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.809999    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d4bc677-6239-47b2-92c1-1924e9cd1b54-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-8bglg\" (UID: \"6d4bc677-6239-47b2-92c1-1924e9cd1b54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.810169 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.810086    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2xzm\" (UniqueName: \"kubernetes.io/projected/6d4bc677-6239-47b2-92c1-1924e9cd1b54-kube-api-access-d2xzm\") pod \"maas-default-gateway-openshift-default-6cc458d887-8bglg\" (UID: \"6d4bc677-6239-47b2-92c1-1924e9cd1b54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.854187 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.854167    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:33.929883 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:33.929855    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:34.112968 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.112889    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:34.258842 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:09:34.258790    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:09:34.273890 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.273867    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"]
Nov 04 02:09:34.276549 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:09:34.276523    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d4bc677_6239_47b2_92c1_1924e9cd1b54.slice/crio-5468aacb53abac4d41818699e66053b4a0c4fd540057f9f0faba1af968955a73 WatchSource:0}: Error finding container 5468aacb53abac4d41818699e66053b4a0c4fd540057f9f0faba1af968955a73: Status 404 returned error can't find the container with id 5468aacb53abac4d41818699e66053b4a0c4fd540057f9f0faba1af968955a73
Nov 04 02:09:34.494712 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.494644    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv"]
Nov 04 02:09:34.497725 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.497704    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv"
Nov 04 02:09:34.526531 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.526503    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2"]
Nov 04 02:09:34.528948 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.528931    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2"
Nov 04 02:09:34.550308 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.550286    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-lhfcf\""
Nov 04 02:09:34.550587 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.550449    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Nov 04 02:09:34.550587 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.550451    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Nov 04 02:09:34.555276 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.555256    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/40a95438-ac03-4f2d-8bc1-8c7dba6403fe-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv\" (UID: \"40a95438-ac03-4f2d-8bc1-8c7dba6403fe\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv"
Nov 04 02:09:34.555418 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.555290    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/40a95438-ac03-4f2d-8bc1-8c7dba6403fe-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv\" (UID: \"40a95438-ac03-4f2d-8bc1-8c7dba6403fe\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv"
Nov 04 02:09:34.555418 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.555341    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9s2t\" (UniqueName: \"kubernetes.io/projected/40a95438-ac03-4f2d-8bc1-8c7dba6403fe-kube-api-access-b9s2t\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv\" (UID: \"40a95438-ac03-4f2d-8bc1-8c7dba6403fe\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv"
Nov 04 02:09:34.556703 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.556683    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv"]
Nov 04 02:09:34.559425 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.559403    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s"]
Nov 04 02:09:34.562224 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.562199    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s"
Nov 04 02:09:34.588452 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.588433    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2"]
Nov 04 02:09:34.620842 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.620822    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s"]
Nov 04 02:09:34.651223 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.651207    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-64nfq\""
Nov 04 02:09:34.651223 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.651215    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-lsqxd\""
Nov 04 02:09:34.656499 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.656480    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc95j\" (UniqueName: \"kubernetes.io/projected/65960d9d-c32a-47ee-b21e-487c51ffa1fb-kube-api-access-fc95j\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2\" (UID: \"65960d9d-c32a-47ee-b21e-487c51ffa1fb\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2"
Nov 04 02:09:34.656564 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.656507    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/65960d9d-c32a-47ee-b21e-487c51ffa1fb-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2\" (UID: \"65960d9d-c32a-47ee-b21e-487c51ffa1fb\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2"
Nov 04 02:09:34.656564 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.656532    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/65960d9d-c32a-47ee-b21e-487c51ffa1fb-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2\" (UID: \"65960d9d-c32a-47ee-b21e-487c51ffa1fb\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2"
Nov 04 02:09:34.656564 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.656548    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s\" (UID: \"bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s"
Nov 04 02:09:34.656725 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.656705    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/40a95438-ac03-4f2d-8bc1-8c7dba6403fe-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv\" (UID: \"40a95438-ac03-4f2d-8bc1-8c7dba6403fe\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv"
Nov 04 02:09:34.656765 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.656751    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/40a95438-ac03-4f2d-8bc1-8c7dba6403fe-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv\" (UID: \"40a95438-ac03-4f2d-8bc1-8c7dba6403fe\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv"
Nov 04 02:09:34.656805 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.656792    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnq4w\" (UniqueName: \"kubernetes.io/projected/bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3-kube-api-access-gnq4w\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s\" (UID: \"bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s"
Nov 04 02:09:34.656867 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.656851    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s\" (UID: \"bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s"
Nov 04 02:09:34.656918 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.656903    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9s2t\" (UniqueName: \"kubernetes.io/projected/40a95438-ac03-4f2d-8bc1-8c7dba6403fe-kube-api-access-b9s2t\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv\" (UID: \"40a95438-ac03-4f2d-8bc1-8c7dba6403fe\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv"
Nov 04 02:09:34.657023 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.657005    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/40a95438-ac03-4f2d-8bc1-8c7dba6403fe-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv\" (UID: \"40a95438-ac03-4f2d-8bc1-8c7dba6403fe\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv"
Nov 04 02:09:34.657081 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.657051    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/40a95438-ac03-4f2d-8bc1-8c7dba6403fe-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv\" (UID: \"40a95438-ac03-4f2d-8bc1-8c7dba6403fe\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv"
Nov 04 02:09:34.717573 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.717549    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9s2t\" (UniqueName: \"kubernetes.io/projected/40a95438-ac03-4f2d-8bc1-8c7dba6403fe-kube-api-access-b9s2t\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv\" (UID: \"40a95438-ac03-4f2d-8bc1-8c7dba6403fe\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv"
Nov 04 02:09:34.758231 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.758204    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnq4w\" (UniqueName: \"kubernetes.io/projected/bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3-kube-api-access-gnq4w\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s\" (UID: \"bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s"
Nov 04 02:09:34.758324 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.758241    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s\" (UID: \"bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s"
Nov 04 02:09:34.758324 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.758311    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc95j\" (UniqueName: \"kubernetes.io/projected/65960d9d-c32a-47ee-b21e-487c51ffa1fb-kube-api-access-fc95j\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2\" (UID: \"65960d9d-c32a-47ee-b21e-487c51ffa1fb\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2"
Nov 04 02:09:34.758460 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.758338    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/65960d9d-c32a-47ee-b21e-487c51ffa1fb-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2\" (UID: \"65960d9d-c32a-47ee-b21e-487c51ffa1fb\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2"
Nov 04 02:09:34.758460 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.758377    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/65960d9d-c32a-47ee-b21e-487c51ffa1fb-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2\" (UID: \"65960d9d-c32a-47ee-b21e-487c51ffa1fb\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2"
Nov 04 02:09:34.758460 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.758406    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s\" (UID: \"bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s"
Nov 04 02:09:34.758703 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.758682    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s\" (UID: \"bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s"
Nov 04 02:09:34.758768 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.758718    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s\" (UID: \"bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s"
Nov 04 02:09:34.758819 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.758796    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/65960d9d-c32a-47ee-b21e-487c51ffa1fb-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2\" (UID: \"65960d9d-c32a-47ee-b21e-487c51ffa1fb\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2"
Nov 04 02:09:34.758854 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.758816    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/65960d9d-c32a-47ee-b21e-487c51ffa1fb-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2\" (UID: \"65960d9d-c32a-47ee-b21e-487c51ffa1fb\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2"
Nov 04 02:09:34.810252 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.810233    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv"
Nov 04 02:09:34.818636 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.818614    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnq4w\" (UniqueName: \"kubernetes.io/projected/bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3-kube-api-access-gnq4w\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s\" (UID: \"bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s"
Nov 04 02:09:34.818712 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.818662    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc95j\" (UniqueName: \"kubernetes.io/projected/65960d9d-c32a-47ee-b21e-487c51ffa1fb-kube-api-access-fc95j\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2\" (UID: \"65960d9d-c32a-47ee-b21e-487c51ffa1fb\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2"
Nov 04 02:09:34.840131 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.840086    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2"
Nov 04 02:09:34.874115 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:34.874060    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s"
Nov 04 02:09:35.229499 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:35.229465    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv"]
Nov 04 02:09:35.232164 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:09:35.232124    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40a95438_ac03_4f2d_8bc1_8c7dba6403fe.slice/crio-43d81406692f6313ca4549786146c960d8f12e1ee8d5dbbce1a9d37dd9a4cf5d WatchSource:0}: Error finding container 43d81406692f6313ca4549786146c960d8f12e1ee8d5dbbce1a9d37dd9a4cf5d: Status 404 returned error can't find the container with id 43d81406692f6313ca4549786146c960d8f12e1ee8d5dbbce1a9d37dd9a4cf5d
Nov 04 02:09:35.235138 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:35.235113    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg" event={"ID":"6d4bc677-6239-47b2-92c1-1924e9cd1b54","Type":"ContainerStarted","Data":"5468aacb53abac4d41818699e66053b4a0c4fd540057f9f0faba1af968955a73"}
Nov 04 02:09:35.279203 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:35.279185    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2"]
Nov 04 02:09:35.281404 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:09:35.281378    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65960d9d_c32a_47ee_b21e_487c51ffa1fb.slice/crio-bffebb1ef59985a3517677402dd12d4cabf2a0999b2c93d86fff14e8e0e06718 WatchSource:0}: Error finding container bffebb1ef59985a3517677402dd12d4cabf2a0999b2c93d86fff14e8e0e06718: Status 404 returned error can't find the container with id bffebb1ef59985a3517677402dd12d4cabf2a0999b2c93d86fff14e8e0e06718
Nov 04 02:09:35.331177 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:35.331140    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s"]
Nov 04 02:09:35.333420 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:09:35.333395    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc4bb9a0_96d3_4355_9bd8_040a61ecc1e3.slice/crio-143806919e8ab9fbf97f0b82c786e7da1ac942a770555dddd225b83d3ab498bf WatchSource:0}: Error finding container 143806919e8ab9fbf97f0b82c786e7da1ac942a770555dddd225b83d3ab498bf: Status 404 returned error can't find the container with id 143806919e8ab9fbf97f0b82c786e7da1ac942a770555dddd225b83d3ab498bf
Nov 04 02:09:36.244479 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:36.244440    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv" event={"ID":"40a95438-ac03-4f2d-8bc1-8c7dba6403fe","Type":"ContainerStarted","Data":"43d81406692f6313ca4549786146c960d8f12e1ee8d5dbbce1a9d37dd9a4cf5d"}
Nov 04 02:09:36.245934 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:36.245899    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s" event={"ID":"bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3","Type":"ContainerStarted","Data":"143806919e8ab9fbf97f0b82c786e7da1ac942a770555dddd225b83d3ab498bf"}
Nov 04 02:09:36.248942 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:36.248532    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2" event={"ID":"65960d9d-c32a-47ee-b21e-487c51ffa1fb","Type":"ContainerStarted","Data":"bffebb1ef59985a3517677402dd12d4cabf2a0999b2c93d86fff14e8e0e06718"}
Nov 04 02:09:37.733791 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:37.733748    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:40.238652 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:40.238619    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:09:40.239060 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:40.238702    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:09:40.239060 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:40.238751    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:09:41.282473 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:41.282435    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg" event={"ID":"6d4bc677-6239-47b2-92c1-1924e9cd1b54","Type":"ContainerStarted","Data":"538f64ee0995a627da78a3bc69f644e9a8c8c29932931154e19223d2ce055269"}
Nov 04 02:09:41.454037 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:41.453989    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg" podStartSLOduration=2.494764443 podStartE2EDuration="8.453974539s" totalImagesPullingTime="5.959210096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:09:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:09:41.452264315 +0000 UTC m=+217.378486960" watchObservedRunningTime="2026-11-04 02:09:41.453974539 +0000 UTC m=+217.380197251"
Nov 04 02:09:41.854745 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:41.854709    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:41.860846 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:41.860816    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:42.288122 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:42.288046    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:42.290446 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:42.290401    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:09:44.259089 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:44.259057    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:44.296782 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:44.296752    4969 generic.go:356] "Generic (PLEG): container finished" podID="65960d9d-c32a-47ee-b21e-487c51ffa1fb" containerID="a608909262b6b4558feff382cf0fe8171a6b36fa29367cb27603fc367f54ca02" exitCode=0
Nov 04 02:09:44.296909 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:44.296836    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2" event={"ID":"65960d9d-c32a-47ee-b21e-487c51ffa1fb","Type":"ContainerDied","Data":"a608909262b6b4558feff382cf0fe8171a6b36fa29367cb27603fc367f54ca02"}
Nov 04 02:09:44.298360 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:44.298334    4969 generic.go:356] "Generic (PLEG): container finished" podID="40a95438-ac03-4f2d-8bc1-8c7dba6403fe" containerID="518f3c2c641475c5e75a6714e4867fc0059c5bb59a3b02caacf5cf24626d150c" exitCode=0
Nov 04 02:09:44.298469 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:44.298359    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv" event={"ID":"40a95438-ac03-4f2d-8bc1-8c7dba6403fe","Type":"ContainerDied","Data":"518f3c2c641475c5e75a6714e4867fc0059c5bb59a3b02caacf5cf24626d150c"}
Nov 04 02:09:44.300136 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:44.300048    4969 generic.go:356] "Generic (PLEG): container finished" podID="bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3" containerID="5a1f6afd938fc739ddf592506930e61c7c963f50c2f4c7b0d4f91b920ee89f52" exitCode=0
Nov 04 02:09:44.300217 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:44.300132    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s" event={"ID":"bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3","Type":"ContainerDied","Data":"5a1f6afd938fc739ddf592506930e61c7c963f50c2f4c7b0d4f91b920ee89f52"}
Nov 04 02:09:44.441619 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:44.441595    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:46.309591 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:46.309520    4969 generic.go:356] "Generic (PLEG): container finished" podID="65960d9d-c32a-47ee-b21e-487c51ffa1fb" containerID="094a7159bfa7dfa06b525011b2358b34e69e63ad2b0a946ee7221586d81c06d6" exitCode=0
Nov 04 02:09:46.309591 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:46.309549    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2" event={"ID":"65960d9d-c32a-47ee-b21e-487c51ffa1fb","Type":"ContainerDied","Data":"094a7159bfa7dfa06b525011b2358b34e69e63ad2b0a946ee7221586d81c06d6"}
Nov 04 02:09:46.311557 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:46.311535    4969 generic.go:356] "Generic (PLEG): container finished" podID="bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3" containerID="26a27f5108c98f733e47cd10fd0163056eff89b922380124c83b6b671f661dfd" exitCode=0
Nov 04 02:09:46.311768 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:46.311562    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s" event={"ID":"bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3","Type":"ContainerDied","Data":"26a27f5108c98f733e47cd10fd0163056eff89b922380124c83b6b671f661dfd"}
Nov 04 02:09:47.204082 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:47.204056    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:47.319338 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:47.319276    4969 generic.go:356] "Generic (PLEG): container finished" podID="40a95438-ac03-4f2d-8bc1-8c7dba6403fe" containerID="6ca8653c55a2bda55dc96151cb5cba7d430f1d54208ca7ca6a611ff72e3baaeb" exitCode=0
Nov 04 02:09:47.319756 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:47.319363    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv" event={"ID":"40a95438-ac03-4f2d-8bc1-8c7dba6403fe","Type":"ContainerDied","Data":"6ca8653c55a2bda55dc96151cb5cba7d430f1d54208ca7ca6a611ff72e3baaeb"}
Nov 04 02:09:47.388345 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:47.388265    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:55.096417 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:55.096378    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:09:55.364772 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:55.364735    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv" event={"ID":"40a95438-ac03-4f2d-8bc1-8c7dba6403fe","Type":"ContainerStarted","Data":"e6ef2edd95741ac786e55be75cc24c7c850ea5be6d2fc16011a7c8bf16c8f18d"}
Nov 04 02:09:55.366850 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:55.366828    4969 generic.go:356] "Generic (PLEG): container finished" podID="bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3" containerID="f39aeac5724827c7ee9226e8e0dc3a11706ed84d2359790a4d48b9634cac9523" exitCode=0
Nov 04 02:09:55.366955 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:55.366914    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s" event={"ID":"bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3","Type":"ContainerDied","Data":"f39aeac5724827c7ee9226e8e0dc3a11706ed84d2359790a4d48b9634cac9523"}
Nov 04 02:09:55.368837 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:55.368818    4969 generic.go:356] "Generic (PLEG): container finished" podID="65960d9d-c32a-47ee-b21e-487c51ffa1fb" containerID="d8afebdfb1574d88d7dde46719cafecde22f02ec1774327f6b6cf0849e26bf49" exitCode=0
Nov 04 02:09:55.368922 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:55.368868    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2" event={"ID":"65960d9d-c32a-47ee-b21e-487c51ffa1fb","Type":"ContainerDied","Data":"d8afebdfb1574d88d7dde46719cafecde22f02ec1774327f6b6cf0849e26bf49"}
Nov 04 02:09:55.521238 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:55.520422    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv" podStartSLOduration=2.367334149 podStartE2EDuration="21.52040399s" totalImagesPullingTime="19.153069841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:09:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:09:55.520285942 +0000 UTC m=+231.446508577" watchObservedRunningTime="2026-11-04 02:09:55.52040399 +0000 UTC m=+231.446626623"
Nov 04 02:09:56.375752 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:56.375715    4969 generic.go:356] "Generic (PLEG): container finished" podID="40a95438-ac03-4f2d-8bc1-8c7dba6403fe" containerID="e6ef2edd95741ac786e55be75cc24c7c850ea5be6d2fc16011a7c8bf16c8f18d" exitCode=0
Nov 04 02:09:56.376224 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:56.375806    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv" event={"ID":"40a95438-ac03-4f2d-8bc1-8c7dba6403fe","Type":"ContainerDied","Data":"e6ef2edd95741ac786e55be75cc24c7c850ea5be6d2fc16011a7c8bf16c8f18d"}
Nov 04 02:09:56.531315 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:56.531292    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2"
Nov 04 02:09:56.555299 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:56.555278    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s"
Nov 04 02:09:56.658154 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:56.658079    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3-bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3-bundle\") pod \"bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3\" (UID: \"bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3\") "
Nov 04 02:09:56.658154 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:56.658127    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/65960d9d-c32a-47ee-b21e-487c51ffa1fb-bundle\" (UniqueName: \"kubernetes.io/empty-dir/65960d9d-c32a-47ee-b21e-487c51ffa1fb-bundle\") pod \"65960d9d-c32a-47ee-b21e-487c51ffa1fb\" (UID: \"65960d9d-c32a-47ee-b21e-487c51ffa1fb\") "
Nov 04 02:09:56.658154 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:56.658144    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/65960d9d-c32a-47ee-b21e-487c51ffa1fb-util\" (UniqueName: \"kubernetes.io/empty-dir/65960d9d-c32a-47ee-b21e-487c51ffa1fb-util\") pod \"65960d9d-c32a-47ee-b21e-487c51ffa1fb\" (UID: \"65960d9d-c32a-47ee-b21e-487c51ffa1fb\") "
Nov 04 02:09:56.658382 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:56.658159    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3-util\" (UniqueName: \"kubernetes.io/empty-dir/bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3-util\") pod \"bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3\" (UID: \"bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3\") "
Nov 04 02:09:56.658382 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:56.658180    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/65960d9d-c32a-47ee-b21e-487c51ffa1fb-kube-api-access-fc95j\" (UniqueName: \"kubernetes.io/projected/65960d9d-c32a-47ee-b21e-487c51ffa1fb-kube-api-access-fc95j\") pod \"65960d9d-c32a-47ee-b21e-487c51ffa1fb\" (UID: \"65960d9d-c32a-47ee-b21e-487c51ffa1fb\") "
Nov 04 02:09:56.658382 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:56.658261    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3-kube-api-access-gnq4w\" (UniqueName: \"kubernetes.io/projected/bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3-kube-api-access-gnq4w\") pod \"bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3\" (UID: \"bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3\") "
Nov 04 02:09:56.658684 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:56.658654    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3-bundle" pod "bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3" (UID: "bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:09:56.658881 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:56.658850    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65960d9d-c32a-47ee-b21e-487c51ffa1fb-bundle" pod "65960d9d-c32a-47ee-b21e-487c51ffa1fb" (UID: "65960d9d-c32a-47ee-b21e-487c51ffa1fb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:09:56.660861 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:56.660837    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3-kube-api-access-gnq4w" pod "bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3" (UID: "bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3"). InnerVolumeSpecName "kube-api-access-gnq4w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:09:56.660985 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:56.660961    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65960d9d-c32a-47ee-b21e-487c51ffa1fb-kube-api-access-fc95j" pod "65960d9d-c32a-47ee-b21e-487c51ffa1fb" (UID: "65960d9d-c32a-47ee-b21e-487c51ffa1fb"). InnerVolumeSpecName "kube-api-access-fc95j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:09:56.662953 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:56.662932    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3-util" pod "bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3" (UID: "bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:09:56.663271 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:56.663251    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65960d9d-c32a-47ee-b21e-487c51ffa1fb-util" pod "65960d9d-c32a-47ee-b21e-487c51ffa1fb" (UID: "65960d9d-c32a-47ee-b21e-487c51ffa1fb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:09:56.759350 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:56.759331    4969 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/65960d9d-c32a-47ee-b21e-487c51ffa1fb-bundle\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:09:56.759350 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:56.759351    4969 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/65960d9d-c32a-47ee-b21e-487c51ffa1fb-util\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:09:56.759467 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:56.759360    4969 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3-util\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:09:56.759467 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:56.759369    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fc95j\" (UniqueName: \"kubernetes.io/projected/65960d9d-c32a-47ee-b21e-487c51ffa1fb-kube-api-access-fc95j\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:09:56.759467 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:56.759388    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnq4w\" (UniqueName: \"kubernetes.io/projected/bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3-kube-api-access-gnq4w\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:09:56.759467 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:56.759396    4969 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3-bundle\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:09:57.382406 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:57.382381    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2"
Nov 04 02:09:57.382823 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:57.382421    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2hgw2" event={"ID":"65960d9d-c32a-47ee-b21e-487c51ffa1fb","Type":"ContainerDied","Data":"bffebb1ef59985a3517677402dd12d4cabf2a0999b2c93d86fff14e8e0e06718"}
Nov 04 02:09:57.382823 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:57.382466    4969 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bffebb1ef59985a3517677402dd12d4cabf2a0999b2c93d86fff14e8e0e06718"
Nov 04 02:09:57.384492 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:57.384467    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s" event={"ID":"bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3","Type":"ContainerDied","Data":"143806919e8ab9fbf97f0b82c786e7da1ac942a770555dddd225b83d3ab498bf"}
Nov 04 02:09:57.384623 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:57.384502    4969 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="143806919e8ab9fbf97f0b82c786e7da1ac942a770555dddd225b83d3ab498bf"
Nov 04 02:09:57.384685 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:57.384624    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d752d2s"
Nov 04 02:09:57.539242 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:57.539214    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv"
Nov 04 02:09:57.665518 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:57.665455    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40a95438-ac03-4f2d-8bc1-8c7dba6403fe-kube-api-access-b9s2t\" (UniqueName: \"kubernetes.io/projected/40a95438-ac03-4f2d-8bc1-8c7dba6403fe-kube-api-access-b9s2t\") pod \"40a95438-ac03-4f2d-8bc1-8c7dba6403fe\" (UID: \"40a95438-ac03-4f2d-8bc1-8c7dba6403fe\") "
Nov 04 02:09:57.665652 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:57.665519    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40a95438-ac03-4f2d-8bc1-8c7dba6403fe-util\" (UniqueName: \"kubernetes.io/empty-dir/40a95438-ac03-4f2d-8bc1-8c7dba6403fe-util\") pod \"40a95438-ac03-4f2d-8bc1-8c7dba6403fe\" (UID: \"40a95438-ac03-4f2d-8bc1-8c7dba6403fe\") "
Nov 04 02:09:57.665652 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:57.665562    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40a95438-ac03-4f2d-8bc1-8c7dba6403fe-bundle\" (UniqueName: \"kubernetes.io/empty-dir/40a95438-ac03-4f2d-8bc1-8c7dba6403fe-bundle\") pod \"40a95438-ac03-4f2d-8bc1-8c7dba6403fe\" (UID: \"40a95438-ac03-4f2d-8bc1-8c7dba6403fe\") "
Nov 04 02:09:57.666068 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:57.666039    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40a95438-ac03-4f2d-8bc1-8c7dba6403fe-bundle" pod "40a95438-ac03-4f2d-8bc1-8c7dba6403fe" (UID: "40a95438-ac03-4f2d-8bc1-8c7dba6403fe"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:09:57.667554 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:57.667532    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40a95438-ac03-4f2d-8bc1-8c7dba6403fe-kube-api-access-b9s2t" pod "40a95438-ac03-4f2d-8bc1-8c7dba6403fe" (UID: "40a95438-ac03-4f2d-8bc1-8c7dba6403fe"). InnerVolumeSpecName "kube-api-access-b9s2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:09:57.673342 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:57.673318    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40a95438-ac03-4f2d-8bc1-8c7dba6403fe-util" pod "40a95438-ac03-4f2d-8bc1-8c7dba6403fe" (UID: "40a95438-ac03-4f2d-8bc1-8c7dba6403fe"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:09:57.766864 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:57.766842    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9s2t\" (UniqueName: \"kubernetes.io/projected/40a95438-ac03-4f2d-8bc1-8c7dba6403fe-kube-api-access-b9s2t\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:09:57.766864 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:57.766865    4969 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/40a95438-ac03-4f2d-8bc1-8c7dba6403fe-util\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:09:57.766989 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:57.766876    4969 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/40a95438-ac03-4f2d-8bc1-8c7dba6403fe-bundle\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:09:58.390293 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:58.390259    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv" event={"ID":"40a95438-ac03-4f2d-8bc1-8c7dba6403fe","Type":"ContainerDied","Data":"43d81406692f6313ca4549786146c960d8f12e1ee8d5dbbce1a9d37dd9a4cf5d"}
Nov 04 02:09:58.390705 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:58.390320    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vt8bv"
Nov 04 02:09:58.390705 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:09:58.390325    4969 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="43d81406692f6313ca4549786146c960d8f12e1ee8d5dbbce1a9d37dd9a4cf5d"
Nov 04 02:10:04.260864 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:10:04.260825    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:10:04.523970 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:04.523887    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:04.699335 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:04.699298    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:13.250303 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:13.250269    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:14.241983 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:14.241940    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:16.247618 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:16.247578    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:16.574170 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:16.574144    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:17.650565 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.650523    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:17.872071 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.872037    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v"]
Nov 04 02:10:17.872394 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.872380    4969 cpu_manager.go:414] "containerMap: removing container" podUID="bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3" containerName="pull"
Nov 04 02:10:17.872438 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.872396    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3" containerName="pull"
Nov 04 02:10:17.872438 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.872417    4969 cpu_manager.go:414] "containerMap: removing container" podUID="65960d9d-c32a-47ee-b21e-487c51ffa1fb" containerName="util"
Nov 04 02:10:17.872438 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.872423    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="65960d9d-c32a-47ee-b21e-487c51ffa1fb" containerName="util"
Nov 04 02:10:17.872438 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.872433    4969 cpu_manager.go:414] "containerMap: removing container" podUID="bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3" containerName="util"
Nov 04 02:10:17.872438 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.872439    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3" containerName="util"
Nov 04 02:10:17.872588 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.872447    4969 cpu_manager.go:414] "containerMap: removing container" podUID="40a95438-ac03-4f2d-8bc1-8c7dba6403fe" containerName="pull"
Nov 04 02:10:17.872588 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.872451    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40a95438-ac03-4f2d-8bc1-8c7dba6403fe" containerName="pull"
Nov 04 02:10:17.872588 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.872466    4969 cpu_manager.go:414] "containerMap: removing container" podUID="65960d9d-c32a-47ee-b21e-487c51ffa1fb" containerName="extract"
Nov 04 02:10:17.872588 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.872470    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="65960d9d-c32a-47ee-b21e-487c51ffa1fb" containerName="extract"
Nov 04 02:10:17.872588 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.872476    4969 cpu_manager.go:414] "containerMap: removing container" podUID="40a95438-ac03-4f2d-8bc1-8c7dba6403fe" containerName="extract"
Nov 04 02:10:17.872588 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.872485    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40a95438-ac03-4f2d-8bc1-8c7dba6403fe" containerName="extract"
Nov 04 02:10:17.872588 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.872494    4969 cpu_manager.go:414] "containerMap: removing container" podUID="40a95438-ac03-4f2d-8bc1-8c7dba6403fe" containerName="util"
Nov 04 02:10:17.872588 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.872498    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40a95438-ac03-4f2d-8bc1-8c7dba6403fe" containerName="util"
Nov 04 02:10:17.872588 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.872504    4969 cpu_manager.go:414] "containerMap: removing container" podUID="65960d9d-c32a-47ee-b21e-487c51ffa1fb" containerName="pull"
Nov 04 02:10:17.872588 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.872507    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="65960d9d-c32a-47ee-b21e-487c51ffa1fb" containerName="pull"
Nov 04 02:10:17.872588 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.872514    4969 cpu_manager.go:414] "containerMap: removing container" podUID="bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3" containerName="extract"
Nov 04 02:10:17.872588 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.872518    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3" containerName="extract"
Nov 04 02:10:17.872588 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.872578    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="65960d9d-c32a-47ee-b21e-487c51ffa1fb" containerName="extract"
Nov 04 02:10:17.872588 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.872585    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="40a95438-ac03-4f2d-8bc1-8c7dba6403fe" containerName="extract"
Nov 04 02:10:17.872588 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.872592    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="bc4bb9a0-96d3-4355-9bd8-040a61ecc1e3" containerName="extract"
Nov 04 02:10:17.874702 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.874687    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v"
Nov 04 02:10:17.918823 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.918759    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s7sj\" (UniqueName: \"kubernetes.io/projected/e5284c4e-9704-4a72-9e0b-97b787a33c39-kube-api-access-7s7sj\") pod \"opendatahub-operator-controller-manager-647c5cc759-4jk6v\" (UID: \"e5284c4e-9704-4a72-9e0b-97b787a33c39\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v"
Nov 04 02:10:17.918823 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.918798    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e5284c4e-9704-4a72-9e0b-97b787a33c39-apiservice-cert\") pod \"opendatahub-operator-controller-manager-647c5cc759-4jk6v\" (UID: \"e5284c4e-9704-4a72-9e0b-97b787a33c39\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v"
Nov 04 02:10:17.918823 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.918818    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5284c4e-9704-4a72-9e0b-97b787a33c39-webhook-cert\") pod \"opendatahub-operator-controller-manager-647c5cc759-4jk6v\" (UID: \"e5284c4e-9704-4a72-9e0b-97b787a33c39\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v"
Nov 04 02:10:17.919070 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.918950    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e5284c4e-9704-4a72-9e0b-97b787a33c39-metrics-cert\") pod \"opendatahub-operator-controller-manager-647c5cc759-4jk6v\" (UID: \"e5284c4e-9704-4a72-9e0b-97b787a33c39\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v"
Nov 04 02:10:17.919070 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.919018    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5284c4e-9704-4a72-9e0b-97b787a33c39-tmp\") pod \"opendatahub-operator-controller-manager-647c5cc759-4jk6v\" (UID: \"e5284c4e-9704-4a72-9e0b-97b787a33c39\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v"
Nov 04 02:10:17.919181 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.919111    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e5284c4e-9704-4a72-9e0b-97b787a33c39-manifests\") pod \"opendatahub-operator-controller-manager-647c5cc759-4jk6v\" (UID: \"e5284c4e-9704-4a72-9e0b-97b787a33c39\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v"
Nov 04 02:10:17.970810 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:17.970787    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:18.002459 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:18.002428    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-6rb58\""
Nov 04 02:10:18.002612 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:18.002586    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 04 02:10:18.002677 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:18.002611    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 04 02:10:18.002677 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:18.002623    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 04 02:10:18.002677 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:18.002658    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 04 02:10:18.002677 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:18.002660    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 04 02:10:18.019668 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:18.019615    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5284c4e-9704-4a72-9e0b-97b787a33c39-tmp\") pod \"opendatahub-operator-controller-manager-647c5cc759-4jk6v\" (UID: \"e5284c4e-9704-4a72-9e0b-97b787a33c39\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v"
Nov 04 02:10:18.019813 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:18.019679    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e5284c4e-9704-4a72-9e0b-97b787a33c39-manifests\") pod \"opendatahub-operator-controller-manager-647c5cc759-4jk6v\" (UID: \"e5284c4e-9704-4a72-9e0b-97b787a33c39\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v"
Nov 04 02:10:18.019813 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:18.019706    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s7sj\" (UniqueName: \"kubernetes.io/projected/e5284c4e-9704-4a72-9e0b-97b787a33c39-kube-api-access-7s7sj\") pod \"opendatahub-operator-controller-manager-647c5cc759-4jk6v\" (UID: \"e5284c4e-9704-4a72-9e0b-97b787a33c39\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v"
Nov 04 02:10:18.019813 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:18.019743    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e5284c4e-9704-4a72-9e0b-97b787a33c39-apiservice-cert\") pod \"opendatahub-operator-controller-manager-647c5cc759-4jk6v\" (UID: \"e5284c4e-9704-4a72-9e0b-97b787a33c39\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v"
Nov 04 02:10:18.019813 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:18.019792    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5284c4e-9704-4a72-9e0b-97b787a33c39-webhook-cert\") pod \"opendatahub-operator-controller-manager-647c5cc759-4jk6v\" (UID: \"e5284c4e-9704-4a72-9e0b-97b787a33c39\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v"
Nov 04 02:10:18.020031 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:18.019844    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e5284c4e-9704-4a72-9e0b-97b787a33c39-metrics-cert\") pod \"opendatahub-operator-controller-manager-647c5cc759-4jk6v\" (UID: \"e5284c4e-9704-4a72-9e0b-97b787a33c39\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v"
Nov 04 02:10:18.020396 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:18.020360    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e5284c4e-9704-4a72-9e0b-97b787a33c39-manifests\") pod \"opendatahub-operator-controller-manager-647c5cc759-4jk6v\" (UID: \"e5284c4e-9704-4a72-9e0b-97b787a33c39\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v"
Nov 04 02:10:18.022691 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:18.022660    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e5284c4e-9704-4a72-9e0b-97b787a33c39-metrics-cert\") pod \"opendatahub-operator-controller-manager-647c5cc759-4jk6v\" (UID: \"e5284c4e-9704-4a72-9e0b-97b787a33c39\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v"
Nov 04 02:10:18.022801 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:18.022786    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5284c4e-9704-4a72-9e0b-97b787a33c39-tmp\") pod \"opendatahub-operator-controller-manager-647c5cc759-4jk6v\" (UID: \"e5284c4e-9704-4a72-9e0b-97b787a33c39\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v"
Nov 04 02:10:18.023166 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:18.023139    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5284c4e-9704-4a72-9e0b-97b787a33c39-webhook-cert\") pod \"opendatahub-operator-controller-manager-647c5cc759-4jk6v\" (UID: \"e5284c4e-9704-4a72-9e0b-97b787a33c39\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v"
Nov 04 02:10:18.023623 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:18.023600    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e5284c4e-9704-4a72-9e0b-97b787a33c39-apiservice-cert\") pod \"opendatahub-operator-controller-manager-647c5cc759-4jk6v\" (UID: \"e5284c4e-9704-4a72-9e0b-97b787a33c39\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v"
Nov 04 02:10:18.081069 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:18.081034    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s7sj\" (UniqueName: \"kubernetes.io/projected/e5284c4e-9704-4a72-9e0b-97b787a33c39-kube-api-access-7s7sj\") pod \"opendatahub-operator-controller-manager-647c5cc759-4jk6v\" (UID: \"e5284c4e-9704-4a72-9e0b-97b787a33c39\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v"
Nov 04 02:10:18.121710 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:18.121684    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v"]
Nov 04 02:10:18.147635 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:18.147610    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:18.190661 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:18.190609    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v"
Nov 04 02:10:18.330621 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:18.330587    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:18.829788 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:18.829765    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v"]
Nov 04 02:10:18.831942 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:10:18.831913    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5284c4e_9704_4a72_9e0b_97b787a33c39.slice/crio-2f2934a83082e8b2bfa6f31eed9b506e4cdeaf4a0c49e999c54c9bb584f0bd07 WatchSource:0}: Error finding container 2f2934a83082e8b2bfa6f31eed9b506e4cdeaf4a0c49e999c54c9bb584f0bd07: Status 404 returned error can't find the container with id 2f2934a83082e8b2bfa6f31eed9b506e4cdeaf4a0c49e999c54c9bb584f0bd07
Nov 04 02:10:19.484162 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:19.484124    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v" event={"ID":"e5284c4e-9704-4a72-9e0b-97b787a33c39","Type":"ContainerStarted","Data":"2f2934a83082e8b2bfa6f31eed9b506e4cdeaf4a0c49e999c54c9bb584f0bd07"}
Nov 04 02:10:20.050914 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:20.050880    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:21.874424 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:21.874383    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:22.396332 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:22.396298    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-dzk6c"]
Nov 04 02:10:22.398886 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:22.398859    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-dzk6c"
Nov 04 02:10:22.451339 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:22.451314    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-d87ff\""
Nov 04 02:10:22.455835 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:22.455798    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95z6w\" (UniqueName: \"kubernetes.io/projected/de6afddf-ef49-434e-a869-4ff600c01ee5-kube-api-access-95z6w\") pod \"dns-operator-controller-manager-8557cbdb84-dzk6c\" (UID: \"de6afddf-ef49-434e-a869-4ff600c01ee5\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-dzk6c"
Nov 04 02:10:22.501028 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:22.500968    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Nov 04 02:10:22.501028 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:22.500970    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Nov 04 02:10:22.501028 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:22.500977    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Nov 04 02:10:22.503698 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:22.503674    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-dzk6c"]
Nov 04 02:10:22.556987 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:22.556956    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95z6w\" (UniqueName: \"kubernetes.io/projected/de6afddf-ef49-434e-a869-4ff600c01ee5-kube-api-access-95z6w\") pod \"dns-operator-controller-manager-8557cbdb84-dzk6c\" (UID: \"de6afddf-ef49-434e-a869-4ff600c01ee5\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-dzk6c"
Nov 04 02:10:22.618004 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:22.617971    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95z6w\" (UniqueName: \"kubernetes.io/projected/de6afddf-ef49-434e-a869-4ff600c01ee5-kube-api-access-95z6w\") pod \"dns-operator-controller-manager-8557cbdb84-dzk6c\" (UID: \"de6afddf-ef49-434e-a869-4ff600c01ee5\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-dzk6c"
Nov 04 02:10:22.712733 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:22.712648    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-dzk6c"
Nov 04 02:10:22.908944 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:22.908905    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:23.108982 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:23.108960    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:23.151453 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:23.151427    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-dzk6c"]
Nov 04 02:10:23.156610 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:10:23.156581    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde6afddf_ef49_434e_a869_4ff600c01ee5.slice/crio-2d052e8c660379a3254334d93e408b02ecc942c74ab07030fc56234444699605 WatchSource:0}: Error finding container 2d052e8c660379a3254334d93e408b02ecc942c74ab07030fc56234444699605: Status 404 returned error can't find the container with id 2d052e8c660379a3254334d93e408b02ecc942c74ab07030fc56234444699605
Nov 04 02:10:23.348416 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:23.348351    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:23.506153 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:23.506120    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-dzk6c" event={"ID":"de6afddf-ef49-434e-a869-4ff600c01ee5","Type":"ContainerStarted","Data":"2d052e8c660379a3254334d93e408b02ecc942c74ab07030fc56234444699605"}
Nov 04 02:10:23.507671 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:23.507641    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v" event={"ID":"e5284c4e-9704-4a72-9e0b-97b787a33c39","Type":"ContainerStarted","Data":"99b2106479d84dd4dd855f32989a370120ff29452e2decb8e003bd7e1b9a7273"}
Nov 04 02:10:24.238635 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:24.238592    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:24.513284 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:24.513255    4969 generic.go:356] "Generic (PLEG): container finished" podID="e5284c4e-9704-4a72-9e0b-97b787a33c39" containerID="99b2106479d84dd4dd855f32989a370120ff29452e2decb8e003bd7e1b9a7273" exitCode=0
Nov 04 02:10:24.513461 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:24.513345    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v" event={"ID":"e5284c4e-9704-4a72-9e0b-97b787a33c39","Type":"ContainerDied","Data":"99b2106479d84dd4dd855f32989a370120ff29452e2decb8e003bd7e1b9a7273"}
Nov 04 02:10:25.876577 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:25.876535    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:26.528038 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:26.527996    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v" event={"ID":"e5284c4e-9704-4a72-9e0b-97b787a33c39","Type":"ContainerStarted","Data":"d08b3d635c184e07cb51e22855ce93531b5570b7639a323fde1ba8b1917ecfcc"}
Nov 04 02:10:26.528860 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:26.528838    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v"
Nov 04 02:10:26.839205 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:26.839084    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v" podStartSLOduration=4.097726235 podStartE2EDuration="9.839067103s" totalImagesPullingTime="5.176219526s" totalInitContainerRuntime="565.121342ms" isStatefulPod=false podCreationTimestamp="2026-11-04 02:10:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:10:26.788268247 +0000 UTC m=+262.714490883" watchObservedRunningTime="2026-11-04 02:10:26.839067103 +0000 UTC m=+262.765289737"
Nov 04 02:10:26.874002 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:26.873968    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:27.060030 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:27.059993    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:28.539155 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:28.539118    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-dzk6c" event={"ID":"de6afddf-ef49-434e-a869-4ff600c01ee5","Type":"ContainerStarted","Data":"e054b60c0ab684d3e0bbc4fe9846b4e759e254e54748dea31c33f73c48feb2f6"}
Nov 04 02:10:28.539522 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:28.539423    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-dzk6c"
Nov 04 02:10:29.191574 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:29.191532    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:29.884517 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:29.884453    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-dzk6c" podStartSLOduration=2.726343672 podStartE2EDuration="7.884439081s" totalImagesPullingTime="5.158095409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:10:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:10:28.655727811 +0000 UTC m=+264.581950447" watchObservedRunningTime="2026-11-04 02:10:29.884439081 +0000 UTC m=+265.810661714"
Nov 04 02:10:29.884901 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:29.884852    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7qdqh"]
Nov 04 02:10:29.888847 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:29.888831    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7qdqh"
Nov 04 02:10:29.917174 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:29.917149    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvvnk\" (UniqueName: \"kubernetes.io/projected/d516e277-83b1-4757-a933-ce24550c5068-kube-api-access-lvvnk\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-7qdqh\" (UID: \"d516e277-83b1-4757-a933-ce24550c5068\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7qdqh"
Nov 04 02:10:29.917276 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:29.917184    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d516e277-83b1-4757-a933-ce24550c5068-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-7qdqh\" (UID: \"d516e277-83b1-4757-a933-ce24550c5068\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7qdqh"
Nov 04 02:10:29.942632 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:29.942607    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-pjt8t\""
Nov 04 02:10:29.942632 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:29.942616    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Nov 04 02:10:29.991912 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:29.991887    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Nov 04 02:10:29.993950 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:29.993930    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7qdqh"]
Nov 04 02:10:30.017824 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:30.017802    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvvnk\" (UniqueName: \"kubernetes.io/projected/d516e277-83b1-4757-a933-ce24550c5068-kube-api-access-lvvnk\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-7qdqh\" (UID: \"d516e277-83b1-4757-a933-ce24550c5068\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7qdqh"
Nov 04 02:10:30.017911 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:30.017834    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d516e277-83b1-4757-a933-ce24550c5068-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-7qdqh\" (UID: \"d516e277-83b1-4757-a933-ce24550c5068\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7qdqh"
Nov 04 02:10:30.018246 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:30.018229    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d516e277-83b1-4757-a933-ce24550c5068-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-7qdqh\" (UID: \"d516e277-83b1-4757-a933-ce24550c5068\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7qdqh"
Nov 04 02:10:30.080996 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:30.080973    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvvnk\" (UniqueName: \"kubernetes.io/projected/d516e277-83b1-4757-a933-ce24550c5068-kube-api-access-lvvnk\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-7qdqh\" (UID: \"d516e277-83b1-4757-a933-ce24550c5068\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7qdqh"
Nov 04 02:10:30.198346 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:30.198299    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7qdqh"
Nov 04 02:10:30.674629 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:30.674604    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7qdqh"]
Nov 04 02:10:30.676895 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:10:30.676863    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd516e277_83b1_4757_a933_ce24550c5068.slice/crio-5366cb4541c613e176010097615c421c7573668f8c4187b63b9acf29251686ab WatchSource:0}: Error finding container 5366cb4541c613e176010097615c421c7573668f8c4187b63b9acf29251686ab: Status 404 returned error can't find the container with id 5366cb4541c613e176010097615c421c7573668f8c4187b63b9acf29251686ab
Nov 04 02:10:30.845175 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:30.845151    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-sc7zj"]
Nov 04 02:10:30.847935 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:30.847920    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-sc7zj"
Nov 04 02:10:30.902076 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:30.902054    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xnwdv\""
Nov 04 02:10:30.910898 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:30.910877    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-sc7zj"]
Nov 04 02:10:30.925382 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:30.925334    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5d54\" (UniqueName: \"kubernetes.io/projected/595c2698-cf72-4c37-8096-bf3dac4c20b7-kube-api-access-h5d54\") pod \"kuadrant-operator-controller-manager-678d87766f-sc7zj\" (UID: \"595c2698-cf72-4c37-8096-bf3dac4c20b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-sc7zj"
Nov 04 02:10:30.925382 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:30.925373    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/595c2698-cf72-4c37-8096-bf3dac4c20b7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-sc7zj\" (UID: \"595c2698-cf72-4c37-8096-bf3dac4c20b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-sc7zj"
Nov 04 02:10:31.026614 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:31.026587    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5d54\" (UniqueName: \"kubernetes.io/projected/595c2698-cf72-4c37-8096-bf3dac4c20b7-kube-api-access-h5d54\") pod \"kuadrant-operator-controller-manager-678d87766f-sc7zj\" (UID: \"595c2698-cf72-4c37-8096-bf3dac4c20b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-sc7zj"
Nov 04 02:10:31.026706 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:31.026626    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/595c2698-cf72-4c37-8096-bf3dac4c20b7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-sc7zj\" (UID: \"595c2698-cf72-4c37-8096-bf3dac4c20b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-sc7zj"
Nov 04 02:10:31.027001 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:31.026982    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/595c2698-cf72-4c37-8096-bf3dac4c20b7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-sc7zj\" (UID: \"595c2698-cf72-4c37-8096-bf3dac4c20b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-sc7zj"
Nov 04 02:10:31.098705 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:31.098676    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5d54\" (UniqueName: \"kubernetes.io/projected/595c2698-cf72-4c37-8096-bf3dac4c20b7-kube-api-access-h5d54\") pod \"kuadrant-operator-controller-manager-678d87766f-sc7zj\" (UID: \"595c2698-cf72-4c37-8096-bf3dac4c20b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-sc7zj"
Nov 04 02:10:31.159423 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:31.159393    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-sc7zj"
Nov 04 02:10:31.198776 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:31.198718    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:31.553581 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:31.553549    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7qdqh" event={"ID":"d516e277-83b1-4757-a933-ce24550c5068","Type":"ContainerStarted","Data":"5366cb4541c613e176010097615c421c7573668f8c4187b63b9acf29251686ab"}
Nov 04 02:10:31.580653 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:31.580632    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-sc7zj"]
Nov 04 02:10:31.582970 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:10:31.582945    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod595c2698_cf72_4c37_8096_bf3dac4c20b7.slice/crio-d41cb54f220f877bf5a6d9f77f0cd7462798a696859fb095695639213a1e70e4 WatchSource:0}: Error finding container d41cb54f220f877bf5a6d9f77f0cd7462798a696859fb095695639213a1e70e4: Status 404 returned error can't find the container with id d41cb54f220f877bf5a6d9f77f0cd7462798a696859fb095695639213a1e70e4
Nov 04 02:10:32.042470 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:32.042428    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:32.561335 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:32.561291    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-sc7zj" event={"ID":"595c2698-cf72-4c37-8096-bf3dac4c20b7","Type":"ContainerStarted","Data":"d41cb54f220f877bf5a6d9f77f0cd7462798a696859fb095695639213a1e70e4"}
Nov 04 02:10:34.263289 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:10:34.263248    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:10:34.574400 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:34.574302    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7qdqh" event={"ID":"d516e277-83b1-4757-a933-ce24550c5068","Type":"ContainerStarted","Data":"cd86d1fb9ba2dd1dd921d966bef4e7d405b6f8fc7da89e816ad638d9ad63b080"}
Nov 04 02:10:34.794838 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:34.794067    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7qdqh" podStartSLOduration=2.161936684 podStartE2EDuration="5.794047661s" totalImagesPullingTime="3.632110977s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:10:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:10:34.743144769 +0000 UTC m=+270.669367403" watchObservedRunningTime="2026-11-04 02:10:34.794047661 +0000 UTC m=+270.720270295"
Nov 04 02:10:35.120739 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:35.120699    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:36.483728 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:36.483660    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:36.708086 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:36.708056    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:37.082750 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:37.082721    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:37.278492 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:37.278453    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:37.470021 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:37.469940    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:37.591071 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:37.591034    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-sc7zj" event={"ID":"595c2698-cf72-4c37-8096-bf3dac4c20b7","Type":"ContainerStarted","Data":"922c2b1f3508a980ee333381a31a6218deb97b532ae28b7dcd595d946b431ff8"}
Nov 04 02:10:37.591544 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:37.591160    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-sc7zj"
Nov 04 02:10:37.817232 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:37.817182    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-sc7zj" podStartSLOduration=2.672058561 podStartE2EDuration="7.817168329s" totalImagesPullingTime="5.145109768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:10:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:10:37.766154929 +0000 UTC m=+273.692377563" watchObservedRunningTime="2026-11-04 02:10:37.817168329 +0000 UTC m=+273.743390961"
Nov 04 02:10:38.541528 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:38.541499    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-647c5cc759-4jk6v"
Nov 04 02:10:39.065902 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:39.065860    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:39.546632 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:39.546608    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-dzk6c"
Nov 04 02:10:40.010836 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:40.010800    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:40.908757 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:40.908698    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:41.125868 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:41.125838    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:41.375532 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:41.375498    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:41.562851 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:41.562820    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:46.062718 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:46.062682    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:46.242946 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:46.242912    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:47.748126 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:47.748083    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:47.928133 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:47.928064    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:48.319666 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:48.319627    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:48.598561 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:48.598476    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-sc7zj"
Nov 04 02:10:48.691750 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:48.691716    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:49.680450 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:49.680417    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:49.829880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:49.829850    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-8j4gv"]
Nov 04 02:10:49.833248 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:49.833232    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-8j4gv"
Nov 04 02:10:49.905342 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:49.905306    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-8j4gv"]
Nov 04 02:10:49.953017 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:49.952937    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Nov 04 02:10:49.953017 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:49.952947    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-2j5d4\""
Nov 04 02:10:49.953017 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:49.952955    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Nov 04 02:10:49.976181 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:49.976150    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shwch\" (UniqueName: \"kubernetes.io/projected/cbd3c0c0-0431-4bf9-b0d7-4c4a6445807b-kube-api-access-shwch\") pod \"cert-manager-597b95cb84-8j4gv\" (UID: \"cbd3c0c0-0431-4bf9-b0d7-4c4a6445807b\") " pod="cert-manager/cert-manager-597b95cb84-8j4gv"
Nov 04 02:10:49.976294 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:49.976203    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cbd3c0c0-0431-4bf9-b0d7-4c4a6445807b-bound-sa-token\") pod \"cert-manager-597b95cb84-8j4gv\" (UID: \"cbd3c0c0-0431-4bf9-b0d7-4c4a6445807b\") " pod="cert-manager/cert-manager-597b95cb84-8j4gv"
Nov 04 02:10:50.052608 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:50.052573    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:50.076644 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:50.076615    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cbd3c0c0-0431-4bf9-b0d7-4c4a6445807b-bound-sa-token\") pod \"cert-manager-597b95cb84-8j4gv\" (UID: \"cbd3c0c0-0431-4bf9-b0d7-4c4a6445807b\") " pod="cert-manager/cert-manager-597b95cb84-8j4gv"
Nov 04 02:10:50.076766 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:50.076685    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shwch\" (UniqueName: \"kubernetes.io/projected/cbd3c0c0-0431-4bf9-b0d7-4c4a6445807b-kube-api-access-shwch\") pod \"cert-manager-597b95cb84-8j4gv\" (UID: \"cbd3c0c0-0431-4bf9-b0d7-4c4a6445807b\") " pod="cert-manager/cert-manager-597b95cb84-8j4gv"
Nov 04 02:10:50.113009 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:50.112986    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-56b54d794b-mf6t9"]
Nov 04 02:10:50.116405 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:50.116389    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-56b54d794b-mf6t9"
Nov 04 02:10:50.147884 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:50.147863    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shwch\" (UniqueName: \"kubernetes.io/projected/cbd3c0c0-0431-4bf9-b0d7-4c4a6445807b-kube-api-access-shwch\") pod \"cert-manager-597b95cb84-8j4gv\" (UID: \"cbd3c0c0-0431-4bf9-b0d7-4c4a6445807b\") " pod="cert-manager/cert-manager-597b95cb84-8j4gv"
Nov 04 02:10:50.148210 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:50.148190    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cbd3c0c0-0431-4bf9-b0d7-4c4a6445807b-bound-sa-token\") pod \"cert-manager-597b95cb84-8j4gv\" (UID: \"cbd3c0c0-0431-4bf9-b0d7-4c4a6445807b\") " pod="cert-manager/cert-manager-597b95cb84-8j4gv"
Nov 04 02:10:50.153088 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:50.153065    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-8j4gv"
Nov 04 02:10:50.177698 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:50.177675    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f71e7139-be33-407f-ad04-ecd9ee83ea48-tls-certs\") pod \"kube-auth-proxy-56b54d794b-mf6t9\" (UID: \"f71e7139-be33-407f-ad04-ecd9ee83ea48\") " pod="openshift-ingress/kube-auth-proxy-56b54d794b-mf6t9"
Nov 04 02:10:50.177811 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:50.177730    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq78f\" (UniqueName: \"kubernetes.io/projected/f71e7139-be33-407f-ad04-ecd9ee83ea48-kube-api-access-pq78f\") pod \"kube-auth-proxy-56b54d794b-mf6t9\" (UID: \"f71e7139-be33-407f-ad04-ecd9ee83ea48\") " pod="openshift-ingress/kube-auth-proxy-56b54d794b-mf6t9"
Nov 04 02:10:50.177855 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:50.177833    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f71e7139-be33-407f-ad04-ecd9ee83ea48-tmp\") pod \"kube-auth-proxy-56b54d794b-mf6t9\" (UID: \"f71e7139-be33-407f-ad04-ecd9ee83ea48\") " pod="openshift-ingress/kube-auth-proxy-56b54d794b-mf6t9"
Nov 04 02:10:50.183487 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:50.183463    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-56b54d794b-mf6t9"]
Nov 04 02:10:50.258618 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:50.258588    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Nov 04 02:10:50.258736 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:50.258588    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Nov 04 02:10:50.258736 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:50.258634    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-pc4n4\""
Nov 04 02:10:50.279098 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:50.279074    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f71e7139-be33-407f-ad04-ecd9ee83ea48-tmp\") pod \"kube-auth-proxy-56b54d794b-mf6t9\" (UID: \"f71e7139-be33-407f-ad04-ecd9ee83ea48\") " pod="openshift-ingress/kube-auth-proxy-56b54d794b-mf6t9"
Nov 04 02:10:50.279214 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:50.279175    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f71e7139-be33-407f-ad04-ecd9ee83ea48-tls-certs\") pod \"kube-auth-proxy-56b54d794b-mf6t9\" (UID: \"f71e7139-be33-407f-ad04-ecd9ee83ea48\") " pod="openshift-ingress/kube-auth-proxy-56b54d794b-mf6t9"
Nov 04 02:10:50.279262 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:50.279236    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq78f\" (UniqueName: \"kubernetes.io/projected/f71e7139-be33-407f-ad04-ecd9ee83ea48-kube-api-access-pq78f\") pod \"kube-auth-proxy-56b54d794b-mf6t9\" (UID: \"f71e7139-be33-407f-ad04-ecd9ee83ea48\") " pod="openshift-ingress/kube-auth-proxy-56b54d794b-mf6t9"
Nov 04 02:10:50.281295 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:50.281271    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f71e7139-be33-407f-ad04-ecd9ee83ea48-tmp\") pod \"kube-auth-proxy-56b54d794b-mf6t9\" (UID: \"f71e7139-be33-407f-ad04-ecd9ee83ea48\") " pod="openshift-ingress/kube-auth-proxy-56b54d794b-mf6t9"
Nov 04 02:10:50.281497 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:50.281479    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f71e7139-be33-407f-ad04-ecd9ee83ea48-tls-certs\") pod \"kube-auth-proxy-56b54d794b-mf6t9\" (UID: \"f71e7139-be33-407f-ad04-ecd9ee83ea48\") " pod="openshift-ingress/kube-auth-proxy-56b54d794b-mf6t9"
Nov 04 02:10:50.356496 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:50.356474    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq78f\" (UniqueName: \"kubernetes.io/projected/f71e7139-be33-407f-ad04-ecd9ee83ea48-kube-api-access-pq78f\") pod \"kube-auth-proxy-56b54d794b-mf6t9\" (UID: \"f71e7139-be33-407f-ad04-ecd9ee83ea48\") " pod="openshift-ingress/kube-auth-proxy-56b54d794b-mf6t9"
Nov 04 02:10:50.364979 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:50.364957    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:50.428842 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:50.428822    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-56b54d794b-mf6t9"
Nov 04 02:10:50.532257 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:50.532232    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-8j4gv"]
Nov 04 02:10:50.533896 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:10:50.533869    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcbd3c0c0_0431_4bf9_b0d7_4c4a6445807b.slice/crio-e378758b5b64c08be881a088bbd1a7f36330108d7deae1cda3e653a45b511006 WatchSource:0}: Error finding container e378758b5b64c08be881a088bbd1a7f36330108d7deae1cda3e653a45b511006: Status 404 returned error can't find the container with id e378758b5b64c08be881a088bbd1a7f36330108d7deae1cda3e653a45b511006
Nov 04 02:10:50.596408 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:50.596385    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:50.650111 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:50.650072    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-8j4gv" event={"ID":"cbd3c0c0-0431-4bf9-b0d7-4c4a6445807b","Type":"ContainerStarted","Data":"e378758b5b64c08be881a088bbd1a7f36330108d7deae1cda3e653a45b511006"}
Nov 04 02:10:50.821003 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:50.820975    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-56b54d794b-mf6t9"]
Nov 04 02:10:50.823018 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:10:50.822991    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf71e7139_be33_407f_ad04_ecd9ee83ea48.slice/crio-fbcba8808a1969fe2580f55cc3e3e21a66a799b6592efe9cd63ce8a9fe3476df WatchSource:0}: Error finding container fbcba8808a1969fe2580f55cc3e3e21a66a799b6592efe9cd63ce8a9fe3476df: Status 404 returned error can't find the container with id fbcba8808a1969fe2580f55cc3e3e21a66a799b6592efe9cd63ce8a9fe3476df
Nov 04 02:10:50.892734 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:50.892695    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:51.077628 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:51.077551    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:51.585686 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:51.585655    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:51.656167 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:51.656131    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-56b54d794b-mf6t9" event={"ID":"f71e7139-be33-407f-ad04-ecd9ee83ea48","Type":"ContainerStarted","Data":"fbcba8808a1969fe2580f55cc3e3e21a66a799b6592efe9cd63ce8a9fe3476df"}
Nov 04 02:10:51.828405 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:51.828375    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:53.176227 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:53.176165    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:53.352456 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:53.352424    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:54.675011 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:54.674928    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-8j4gv" event={"ID":"cbd3c0c0-0431-4bf9-b0d7-4c4a6445807b","Type":"ContainerStarted","Data":"38b1ced638f0e0cb5ca9ae9910cda63008ae3a96a1afa800142303fe1ff24279"}
Nov 04 02:10:54.676443 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:54.676418    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-56b54d794b-mf6t9" event={"ID":"f71e7139-be33-407f-ad04-ecd9ee83ea48","Type":"ContainerStarted","Data":"dbfd4b85d6ff3bbf4935f01cb0d903ce020241ed397678e3e76a21d440d53e3a"}
Nov 04 02:10:54.790136 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:54.790074    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-8j4gv" podStartSLOduration=1.975920329 podStartE2EDuration="5.790061191s" totalImagesPullingTime="3.814140862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:10:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:10:54.788718309 +0000 UTC m=+290.714940944" watchObservedRunningTime="2026-11-04 02:10:54.790061191 +0000 UTC m=+290.716283802"
Nov 04 02:10:54.908361 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:54.908312    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-56b54d794b-mf6t9" podStartSLOduration=1.618079937 podStartE2EDuration="4.908298361s" totalImagesPullingTime="3.290218424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:10:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:10:54.907395631 +0000 UTC m=+290.833618265" watchObservedRunningTime="2026-11-04 02:10:54.908298361 +0000 UTC m=+290.834520994"
Nov 04 02:10:55.461699 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:55.461625    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:56.204877 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:56.204841    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:56.380242 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:56.380189    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:57.694546 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:57.694515    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:58.558480 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:58.558445    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:58.735529 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:58.735494    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:10:58.915828 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:10:58.915752    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:04.216777 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:04.216746    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jcqmp_16f00163-6ec3-4c10-81e4-c8a501e15e82/console-operator/2.log"
Nov 04 02:11:04.217381 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:04.217356    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jcqmp_16f00163-6ec3-4c10-81e4-c8a501e15e82/console-operator/2.log"
Nov 04 02:11:04.259676 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:11:04.259644    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:11:05.168061 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:05.168028    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:05.342728 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:05.342698    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:10.240579 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:10.240550    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:10.421143 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:10.421086    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:11.047790 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:11.047741    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:12.228319 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:12.228286    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:12.403189 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:12.403159    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:17.349325 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:17.349246    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:17.522385 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:17.522350    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:17.953964 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:17.953928    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:18.129598 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:18.129563    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:20.868412 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:20.868379    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:21.046809 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:21.046778    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:21.351196 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:21.351168    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:21.529537 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:21.529503    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:26.952146 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:26.952088    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:27.123998 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:27.123963    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:33.520145 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:33.520084    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:33.697965 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:33.697934    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:33.998721 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:33.998644    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:34.178655 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:34.178621    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:34.259847 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:11:34.259810    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:11:35.090984 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.090953    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:35.146744 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.146717    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8b95c9c66-sv5bj"]
Nov 04 02:11:35.151192 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.151172    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8b95c9c66-sv5bj"
Nov 04 02:11:35.203597 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.203570    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Nov 04 02:11:35.230730 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.230700    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6t6g\" (UniqueName: \"kubernetes.io/projected/9db636c3-8b81-48c2-8c2e-6271b074112a-kube-api-access-c6t6g\") pod \"maas-controller-8b95c9c66-sv5bj\" (UID: \"9db636c3-8b81-48c2-8c2e-6271b074112a\") " pod="opendatahub/maas-controller-8b95c9c66-sv5bj"
Nov 04 02:11:35.230841 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.230789    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9db636c3-8b81-48c2-8c2e-6271b074112a-metrics-certs\") pod \"maas-controller-8b95c9c66-sv5bj\" (UID: \"9db636c3-8b81-48c2-8c2e-6271b074112a\") " pod="opendatahub/maas-controller-8b95c9c66-sv5bj"
Nov 04 02:11:35.230888 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.230849    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9db636c3-8b81-48c2-8c2e-6271b074112a-webhook-cert\") pod \"maas-controller-8b95c9c66-sv5bj\" (UID: \"9db636c3-8b81-48c2-8c2e-6271b074112a\") " pod="opendatahub/maas-controller-8b95c9c66-sv5bj"
Nov 04 02:11:35.252899 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.252878    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 04 02:11:35.253057 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.252878    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Nov 04 02:11:35.253057 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.252899    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mvwv2\""
Nov 04 02:11:35.253057 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.252878    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Nov 04 02:11:35.253057 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.252913    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 04 02:11:35.254903 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.254884    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8b95c9c66-sv5bj"]
Nov 04 02:11:35.271382 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.271354    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:35.331371 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.331348    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6t6g\" (UniqueName: \"kubernetes.io/projected/9db636c3-8b81-48c2-8c2e-6271b074112a-kube-api-access-c6t6g\") pod \"maas-controller-8b95c9c66-sv5bj\" (UID: \"9db636c3-8b81-48c2-8c2e-6271b074112a\") " pod="opendatahub/maas-controller-8b95c9c66-sv5bj"
Nov 04 02:11:35.331485 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.331417    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9db636c3-8b81-48c2-8c2e-6271b074112a-metrics-certs\") pod \"maas-controller-8b95c9c66-sv5bj\" (UID: \"9db636c3-8b81-48c2-8c2e-6271b074112a\") " pod="opendatahub/maas-controller-8b95c9c66-sv5bj"
Nov 04 02:11:35.331485 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.331458    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9db636c3-8b81-48c2-8c2e-6271b074112a-webhook-cert\") pod \"maas-controller-8b95c9c66-sv5bj\" (UID: \"9db636c3-8b81-48c2-8c2e-6271b074112a\") " pod="opendatahub/maas-controller-8b95c9c66-sv5bj"
Nov 04 02:11:35.331634 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:11:35.331617    4969 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Nov 04 02:11:35.331709 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:11:35.331697    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9db636c3-8b81-48c2-8c2e-6271b074112a-webhook-cert podName:9db636c3-8b81-48c2-8c2e-6271b074112a nodeName:}" failed. No retries permitted until 2026-11-04 02:11:35.831676167 +0000 UTC m=+331.757898795 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/9db636c3-8b81-48c2-8c2e-6271b074112a-webhook-cert") pod "maas-controller-8b95c9c66-sv5bj" (UID: "9db636c3-8b81-48c2-8c2e-6271b074112a") : secret "maas-controller-webhook-cert" not found
Nov 04 02:11:35.333868 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.333839    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9db636c3-8b81-48c2-8c2e-6271b074112a-metrics-certs\") pod \"maas-controller-8b95c9c66-sv5bj\" (UID: \"9db636c3-8b81-48c2-8c2e-6271b074112a\") " pod="opendatahub/maas-controller-8b95c9c66-sv5bj"
Nov 04 02:11:35.392063 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.392011    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6t6g\" (UniqueName: \"kubernetes.io/projected/9db636c3-8b81-48c2-8c2e-6271b074112a-kube-api-access-c6t6g\") pod \"maas-controller-8b95c9c66-sv5bj\" (UID: \"9db636c3-8b81-48c2-8c2e-6271b074112a\") " pod="opendatahub/maas-controller-8b95c9c66-sv5bj"
Nov 04 02:11:35.451990 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.451955    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:35.576419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.576392    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-f759c744c-cr7gd"]
Nov 04 02:11:35.578623 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.578607    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f759c744c-cr7gd"
Nov 04 02:11:35.626264 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.626242    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:35.633443 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.633422    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bedd7c8-c5a9-419d-90bc-de1428f21eb3-metrics-certs\") pod \"maas-controller-f759c744c-cr7gd\" (UID: \"1bedd7c8-c5a9-419d-90bc-de1428f21eb3\") " pod="opendatahub/maas-controller-f759c744c-cr7gd"
Nov 04 02:11:35.633544 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.633460    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1bedd7c8-c5a9-419d-90bc-de1428f21eb3-webhook-cert\") pod \"maas-controller-f759c744c-cr7gd\" (UID: \"1bedd7c8-c5a9-419d-90bc-de1428f21eb3\") " pod="opendatahub/maas-controller-f759c744c-cr7gd"
Nov 04 02:11:35.633544 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.633493    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv5d5\" (UniqueName: \"kubernetes.io/projected/1bedd7c8-c5a9-419d-90bc-de1428f21eb3-kube-api-access-gv5d5\") pod \"maas-controller-f759c744c-cr7gd\" (UID: \"1bedd7c8-c5a9-419d-90bc-de1428f21eb3\") " pod="opendatahub/maas-controller-f759c744c-cr7gd"
Nov 04 02:11:35.644394 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.644341    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f759c744c-cr7gd"]
Nov 04 02:11:35.734108 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.734075    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bedd7c8-c5a9-419d-90bc-de1428f21eb3-metrics-certs\") pod \"maas-controller-f759c744c-cr7gd\" (UID: \"1bedd7c8-c5a9-419d-90bc-de1428f21eb3\") " pod="opendatahub/maas-controller-f759c744c-cr7gd"
Nov 04 02:11:35.734211 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.734138    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1bedd7c8-c5a9-419d-90bc-de1428f21eb3-webhook-cert\") pod \"maas-controller-f759c744c-cr7gd\" (UID: \"1bedd7c8-c5a9-419d-90bc-de1428f21eb3\") " pod="opendatahub/maas-controller-f759c744c-cr7gd"
Nov 04 02:11:35.734211 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.734172    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv5d5\" (UniqueName: \"kubernetes.io/projected/1bedd7c8-c5a9-419d-90bc-de1428f21eb3-kube-api-access-gv5d5\") pod \"maas-controller-f759c744c-cr7gd\" (UID: \"1bedd7c8-c5a9-419d-90bc-de1428f21eb3\") " pod="opendatahub/maas-controller-f759c744c-cr7gd"
Nov 04 02:11:35.736606 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.736583    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bedd7c8-c5a9-419d-90bc-de1428f21eb3-metrics-certs\") pod \"maas-controller-f759c744c-cr7gd\" (UID: \"1bedd7c8-c5a9-419d-90bc-de1428f21eb3\") " pod="opendatahub/maas-controller-f759c744c-cr7gd"
Nov 04 02:11:35.736700 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.736616    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1bedd7c8-c5a9-419d-90bc-de1428f21eb3-webhook-cert\") pod \"maas-controller-f759c744c-cr7gd\" (UID: \"1bedd7c8-c5a9-419d-90bc-de1428f21eb3\") " pod="opendatahub/maas-controller-f759c744c-cr7gd"
Nov 04 02:11:35.794647 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.794623    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv5d5\" (UniqueName: \"kubernetes.io/projected/1bedd7c8-c5a9-419d-90bc-de1428f21eb3-kube-api-access-gv5d5\") pod \"maas-controller-f759c744c-cr7gd\" (UID: \"1bedd7c8-c5a9-419d-90bc-de1428f21eb3\") " pod="opendatahub/maas-controller-f759c744c-cr7gd"
Nov 04 02:11:35.834768 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.834743    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9db636c3-8b81-48c2-8c2e-6271b074112a-webhook-cert\") pod \"maas-controller-8b95c9c66-sv5bj\" (UID: \"9db636c3-8b81-48c2-8c2e-6271b074112a\") " pod="opendatahub/maas-controller-8b95c9c66-sv5bj"
Nov 04 02:11:35.836777 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.836761    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9db636c3-8b81-48c2-8c2e-6271b074112a-webhook-cert\") pod \"maas-controller-8b95c9c66-sv5bj\" (UID: \"9db636c3-8b81-48c2-8c2e-6271b074112a\") " pod="opendatahub/maas-controller-8b95c9c66-sv5bj"
Nov 04 02:11:35.890818 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:35.890792    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f759c744c-cr7gd"
Nov 04 02:11:36.062561 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:36.062529    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8b95c9c66-sv5bj"
Nov 04 02:11:36.259220 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:36.259191    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f759c744c-cr7gd"]
Nov 04 02:11:36.261277 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:11:36.261250    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bedd7c8_c5a9_419d_90bc_de1428f21eb3.slice/crio-89c5cbf3eab61414cfff134de67f8f8b6099ac8a9823b9e274c6d9257115f9ff WatchSource:0}: Error finding container 89c5cbf3eab61414cfff134de67f8f8b6099ac8a9823b9e274c6d9257115f9ff: Status 404 returned error can't find the container with id 89c5cbf3eab61414cfff134de67f8f8b6099ac8a9823b9e274c6d9257115f9ff
Nov 04 02:11:36.262498 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:36.262482    4969 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 02:11:36.431364 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:36.431334    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8b95c9c66-sv5bj"]
Nov 04 02:11:36.433977 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:11:36.433954    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9db636c3_8b81_48c2_8c2e_6271b074112a.slice/crio-074168f46647a957b8ac5bfba07abef0b55b50eacb706b19288ac7b86b20e3a1 WatchSource:0}: Error finding container 074168f46647a957b8ac5bfba07abef0b55b50eacb706b19288ac7b86b20e3a1: Status 404 returned error can't find the container with id 074168f46647a957b8ac5bfba07abef0b55b50eacb706b19288ac7b86b20e3a1
Nov 04 02:11:36.871498 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:36.871458    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8b95c9c66-sv5bj" event={"ID":"9db636c3-8b81-48c2-8c2e-6271b074112a","Type":"ContainerStarted","Data":"074168f46647a957b8ac5bfba07abef0b55b50eacb706b19288ac7b86b20e3a1"}
Nov 04 02:11:36.872683 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:36.872655    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f759c744c-cr7gd" event={"ID":"1bedd7c8-c5a9-419d-90bc-de1428f21eb3","Type":"ContainerStarted","Data":"89c5cbf3eab61414cfff134de67f8f8b6099ac8a9823b9e274c6d9257115f9ff"}
Nov 04 02:11:40.991123 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:40.991076    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:41.162032 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:41.161996    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:41.903017 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:41.902936    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8b95c9c66-sv5bj" event={"ID":"9db636c3-8b81-48c2-8c2e-6271b074112a","Type":"ContainerStarted","Data":"7ea5a6343b8d54676fea2ae1ec1b0cb333843a2bfee7eb06dff16eb99f733b7c"}
Nov 04 02:11:41.903187 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:41.903031    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8b95c9c66-sv5bj"
Nov 04 02:11:41.904817 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:41.904795    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f759c744c-cr7gd" event={"ID":"1bedd7c8-c5a9-419d-90bc-de1428f21eb3","Type":"ContainerStarted","Data":"43c0c7055af355bb804c6f27e25132a49b904d4dcd49213f5f3bc245c81f533c"}
Nov 04 02:11:41.904930 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:41.904920    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-f759c744c-cr7gd"
Nov 04 02:11:41.978407 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:41.978381    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:42.021851 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:42.021812    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-8b95c9c66-sv5bj" podStartSLOduration=1.925727102 podStartE2EDuration="7.021801069s" totalImagesPullingTime="5.096073967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:11:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:11:42.020386563 +0000 UTC m=+337.946609196" watchObservedRunningTime="2026-11-04 02:11:42.021801069 +0000 UTC m=+337.948023719"
Nov 04 02:11:42.165635 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:42.165569    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:42.185192 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:42.185144    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-f759c744c-cr7gd" podStartSLOduration=1.908650663 podStartE2EDuration="7.185126374s" totalImagesPullingTime="5.276475711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:11:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:11:42.183380908 +0000 UTC m=+338.109603557" watchObservedRunningTime="2026-11-04 02:11:42.185126374 +0000 UTC m=+338.111349008"
Nov 04 02:11:45.110497 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:45.110466    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:45.287229 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:45.287195    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:11:52.911672 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:52.911632    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8b95c9c66-sv5bj"
Nov 04 02:11:52.912288 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:52.911910    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-f759c744c-cr7gd"
Nov 04 02:11:53.217055 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:53.216977    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8b95c9c66-sv5bj"]
Nov 04 02:11:53.217245 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:53.217202    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-8b95c9c66-sv5bj" podUID="9db636c3-8b81-48c2-8c2e-6271b074112a" containerName="manager" containerID="cri-o://7ea5a6343b8d54676fea2ae1ec1b0cb333843a2bfee7eb06dff16eb99f733b7c" gracePeriod=30
Nov 04 02:11:53.458612 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:53.458586    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8b95c9c66-sv5bj"
Nov 04 02:11:53.580143 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:53.580109    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9db636c3-8b81-48c2-8c2e-6271b074112a-kube-api-access-c6t6g\" (UniqueName: \"kubernetes.io/projected/9db636c3-8b81-48c2-8c2e-6271b074112a-kube-api-access-c6t6g\") pod \"9db636c3-8b81-48c2-8c2e-6271b074112a\" (UID: \"9db636c3-8b81-48c2-8c2e-6271b074112a\") "
Nov 04 02:11:53.580295 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:53.580214    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9db636c3-8b81-48c2-8c2e-6271b074112a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9db636c3-8b81-48c2-8c2e-6271b074112a-metrics-certs\") pod \"9db636c3-8b81-48c2-8c2e-6271b074112a\" (UID: \"9db636c3-8b81-48c2-8c2e-6271b074112a\") "
Nov 04 02:11:53.580295 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:53.580283    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9db636c3-8b81-48c2-8c2e-6271b074112a-webhook-cert\" (UniqueName: \"kubernetes.io/secret/9db636c3-8b81-48c2-8c2e-6271b074112a-webhook-cert\") pod \"9db636c3-8b81-48c2-8c2e-6271b074112a\" (UID: \"9db636c3-8b81-48c2-8c2e-6271b074112a\") "
Nov 04 02:11:53.582504 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:53.582480    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9db636c3-8b81-48c2-8c2e-6271b074112a-metrics-certs" pod "9db636c3-8b81-48c2-8c2e-6271b074112a" (UID: "9db636c3-8b81-48c2-8c2e-6271b074112a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:11:53.582620 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:53.582570    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9db636c3-8b81-48c2-8c2e-6271b074112a-webhook-cert" pod "9db636c3-8b81-48c2-8c2e-6271b074112a" (UID: "9db636c3-8b81-48c2-8c2e-6271b074112a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:11:53.583189 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:53.583168    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9db636c3-8b81-48c2-8c2e-6271b074112a-kube-api-access-c6t6g" pod "9db636c3-8b81-48c2-8c2e-6271b074112a" (UID: "9db636c3-8b81-48c2-8c2e-6271b074112a"). InnerVolumeSpecName "kube-api-access-c6t6g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:11:53.681356 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:53.681330    4969 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9db636c3-8b81-48c2-8c2e-6271b074112a-metrics-certs\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:11:53.681356 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:53.681354    4969 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9db636c3-8b81-48c2-8c2e-6271b074112a-webhook-cert\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:11:53.681515 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:53.681365    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6t6g\" (UniqueName: \"kubernetes.io/projected/9db636c3-8b81-48c2-8c2e-6271b074112a-kube-api-access-c6t6g\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:11:53.960986 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:53.960897    4969 generic.go:356] "Generic (PLEG): container finished" podID="9db636c3-8b81-48c2-8c2e-6271b074112a" containerID="7ea5a6343b8d54676fea2ae1ec1b0cb333843a2bfee7eb06dff16eb99f733b7c" exitCode=0
Nov 04 02:11:53.960986 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:53.960953    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8b95c9c66-sv5bj" event={"ID":"9db636c3-8b81-48c2-8c2e-6271b074112a","Type":"ContainerDied","Data":"7ea5a6343b8d54676fea2ae1ec1b0cb333843a2bfee7eb06dff16eb99f733b7c"}
Nov 04 02:11:53.961456 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:53.960996    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8b95c9c66-sv5bj" event={"ID":"9db636c3-8b81-48c2-8c2e-6271b074112a","Type":"ContainerDied","Data":"074168f46647a957b8ac5bfba07abef0b55b50eacb706b19288ac7b86b20e3a1"}
Nov 04 02:11:53.961456 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:53.961020    4969 scope.go:122] "RemoveContainer" containerID="7ea5a6343b8d54676fea2ae1ec1b0cb333843a2bfee7eb06dff16eb99f733b7c"
Nov 04 02:11:53.961456 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:53.960956    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8b95c9c66-sv5bj"
Nov 04 02:11:53.970584 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:53.970564    4969 scope.go:122] "RemoveContainer" containerID="7ea5a6343b8d54676fea2ae1ec1b0cb333843a2bfee7eb06dff16eb99f733b7c"
Nov 04 02:11:53.970845 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:11:53.970829    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ea5a6343b8d54676fea2ae1ec1b0cb333843a2bfee7eb06dff16eb99f733b7c\": container with ID starting with 7ea5a6343b8d54676fea2ae1ec1b0cb333843a2bfee7eb06dff16eb99f733b7c not found: ID does not exist" containerID="7ea5a6343b8d54676fea2ae1ec1b0cb333843a2bfee7eb06dff16eb99f733b7c"
Nov 04 02:11:53.970908 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:53.970856    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ea5a6343b8d54676fea2ae1ec1b0cb333843a2bfee7eb06dff16eb99f733b7c"} err="failed to get container status \"7ea5a6343b8d54676fea2ae1ec1b0cb333843a2bfee7eb06dff16eb99f733b7c\": rpc error: code = NotFound desc = could not find container \"7ea5a6343b8d54676fea2ae1ec1b0cb333843a2bfee7eb06dff16eb99f733b7c\": container with ID starting with 7ea5a6343b8d54676fea2ae1ec1b0cb333843a2bfee7eb06dff16eb99f733b7c not found: ID does not exist"
Nov 04 02:11:54.124254 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:54.124220    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8b95c9c66-sv5bj"]
Nov 04 02:11:54.173768 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:54.173732    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-8b95c9c66-sv5bj"]
Nov 04 02:11:54.200754 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:54.200720    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9db636c3-8b81-48c2-8c2e-6271b074112a" path="/var/lib/kubelet/pods/9db636c3-8b81-48c2-8c2e-6271b074112a/volumes"
Nov 04 02:11:55.810601 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:11:55.810564    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:04.259502 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:12:04.259463    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:12:08.169282 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:08.169215    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:08.343771 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:08.343732    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:08.530970 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:08.530930    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:08.707106 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:08.707062    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:08.884429 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:08.884347    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:09.027105 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.027072    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-jp7lh"]
Nov 04 02:12:09.027474 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.027461    4969 cpu_manager.go:414] "containerMap: removing container" podUID="9db636c3-8b81-48c2-8c2e-6271b074112a" containerName="manager"
Nov 04 02:12:09.027518 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.027476    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9db636c3-8b81-48c2-8c2e-6271b074112a" containerName="manager"
Nov 04 02:12:09.027552 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.027539    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="9db636c3-8b81-48c2-8c2e-6271b074112a" containerName="manager"
Nov 04 02:12:09.031924 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.031908    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-jp7lh"
Nov 04 02:12:09.066266 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.066238    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:09.086813 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.086775    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-psf4w\""
Nov 04 02:12:09.086813 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.086787    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Nov 04 02:12:09.088936 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.088910    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-jp7lh"]
Nov 04 02:12:09.125603 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.125579    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32f36ce2-d4d6-4b82-9ffe-ff5796462611-cert\") pod \"kserve-controller-manager-776cc66d47-jp7lh\" (UID: \"32f36ce2-d4d6-4b82-9ffe-ff5796462611\") " pod="opendatahub/kserve-controller-manager-776cc66d47-jp7lh"
Nov 04 02:12:09.125712 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.125621    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fggnm\" (UniqueName: \"kubernetes.io/projected/32f36ce2-d4d6-4b82-9ffe-ff5796462611-kube-api-access-fggnm\") pod \"kserve-controller-manager-776cc66d47-jp7lh\" (UID: \"32f36ce2-d4d6-4b82-9ffe-ff5796462611\") " pod="opendatahub/kserve-controller-manager-776cc66d47-jp7lh"
Nov 04 02:12:09.226779 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.226717    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fggnm\" (UniqueName: \"kubernetes.io/projected/32f36ce2-d4d6-4b82-9ffe-ff5796462611-kube-api-access-fggnm\") pod \"kserve-controller-manager-776cc66d47-jp7lh\" (UID: \"32f36ce2-d4d6-4b82-9ffe-ff5796462611\") " pod="opendatahub/kserve-controller-manager-776cc66d47-jp7lh"
Nov 04 02:12:09.227123 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.226801    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32f36ce2-d4d6-4b82-9ffe-ff5796462611-cert\") pod \"kserve-controller-manager-776cc66d47-jp7lh\" (UID: \"32f36ce2-d4d6-4b82-9ffe-ff5796462611\") " pod="opendatahub/kserve-controller-manager-776cc66d47-jp7lh"
Nov 04 02:12:09.229310 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.229284    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/32f36ce2-d4d6-4b82-9ffe-ff5796462611-cert\") pod \"kserve-controller-manager-776cc66d47-jp7lh\" (UID: \"32f36ce2-d4d6-4b82-9ffe-ff5796462611\") " pod="opendatahub/kserve-controller-manager-776cc66d47-jp7lh"
Nov 04 02:12:09.251144 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.251124    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:09.255757 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.255737    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-52dc4"]
Nov 04 02:12:09.259208 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.259189    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-52dc4"
Nov 04 02:12:09.287569 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.287550    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fggnm\" (UniqueName: \"kubernetes.io/projected/32f36ce2-d4d6-4b82-9ffe-ff5796462611-kube-api-access-fggnm\") pod \"kserve-controller-manager-776cc66d47-jp7lh\" (UID: \"32f36ce2-d4d6-4b82-9ffe-ff5796462611\") " pod="opendatahub/kserve-controller-manager-776cc66d47-jp7lh"
Nov 04 02:12:09.313734 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.313717    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-jfkwf\""
Nov 04 02:12:09.313831 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.313748    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Nov 04 02:12:09.318203 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.318180    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-52dc4"]
Nov 04 02:12:09.344675 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.344645    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-jp7lh"
Nov 04 02:12:09.428865 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.428827    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46cceba3-d979-4401-b4f3-decd198016e8-tls-certs\") pod \"model-serving-api-75bd89fdfc-52dc4\" (UID: \"46cceba3-d979-4401-b4f3-decd198016e8\") " pod="opendatahub/model-serving-api-75bd89fdfc-52dc4"
Nov 04 02:12:09.429008 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.428989    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkhnk\" (UniqueName: \"kubernetes.io/projected/46cceba3-d979-4401-b4f3-decd198016e8-kube-api-access-qkhnk\") pod \"model-serving-api-75bd89fdfc-52dc4\" (UID: \"46cceba3-d979-4401-b4f3-decd198016e8\") " pod="opendatahub/model-serving-api-75bd89fdfc-52dc4"
Nov 04 02:12:09.530063 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.530038    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46cceba3-d979-4401-b4f3-decd198016e8-tls-certs\") pod \"model-serving-api-75bd89fdfc-52dc4\" (UID: \"46cceba3-d979-4401-b4f3-decd198016e8\") " pod="opendatahub/model-serving-api-75bd89fdfc-52dc4"
Nov 04 02:12:09.530195 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.530120    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkhnk\" (UniqueName: \"kubernetes.io/projected/46cceba3-d979-4401-b4f3-decd198016e8-kube-api-access-qkhnk\") pod \"model-serving-api-75bd89fdfc-52dc4\" (UID: \"46cceba3-d979-4401-b4f3-decd198016e8\") " pod="opendatahub/model-serving-api-75bd89fdfc-52dc4"
Nov 04 02:12:09.532456 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.532433    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46cceba3-d979-4401-b4f3-decd198016e8-tls-certs\") pod \"model-serving-api-75bd89fdfc-52dc4\" (UID: \"46cceba3-d979-4401-b4f3-decd198016e8\") " pod="opendatahub/model-serving-api-75bd89fdfc-52dc4"
Nov 04 02:12:09.590591 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.590566    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkhnk\" (UniqueName: \"kubernetes.io/projected/46cceba3-d979-4401-b4f3-decd198016e8-kube-api-access-qkhnk\") pod \"model-serving-api-75bd89fdfc-52dc4\" (UID: \"46cceba3-d979-4401-b4f3-decd198016e8\") " pod="opendatahub/model-serving-api-75bd89fdfc-52dc4"
Nov 04 02:12:09.714174 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.714153    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-jp7lh"]
Nov 04 02:12:09.716189 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:12:09.716162    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32f36ce2_d4d6_4b82_9ffe_ff5796462611.slice/crio-75328b2daf65a1d957812e1fd03fdcf8464ec7df6b560b343467f1d59ddbfa0f WatchSource:0}: Error finding container 75328b2daf65a1d957812e1fd03fdcf8464ec7df6b560b343467f1d59ddbfa0f: Status 404 returned error can't find the container with id 75328b2daf65a1d957812e1fd03fdcf8464ec7df6b560b343467f1d59ddbfa0f
Nov 04 02:12:09.872048 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:09.871978    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-52dc4"
Nov 04 02:12:10.053947 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:10.053910    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-jp7lh" event={"ID":"32f36ce2-d4d6-4b82-9ffe-ff5796462611","Type":"ContainerStarted","Data":"75328b2daf65a1d957812e1fd03fdcf8464ec7df6b560b343467f1d59ddbfa0f"}
Nov 04 02:12:10.242636 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:10.242615    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-52dc4"]
Nov 04 02:12:10.244647 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:12:10.244626    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46cceba3_d979_4401_b4f3_decd198016e8.slice/crio-462aff14f99fa761ce3f77109bc3749bd344fe88bd7407469a6409bee07ac64b WatchSource:0}: Error finding container 462aff14f99fa761ce3f77109bc3749bd344fe88bd7407469a6409bee07ac64b: Status 404 returned error can't find the container with id 462aff14f99fa761ce3f77109bc3749bd344fe88bd7407469a6409bee07ac64b
Nov 04 02:12:10.424915 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:10.424889    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:11.060667 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:11.060631    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-52dc4" event={"ID":"46cceba3-d979-4401-b4f3-decd198016e8","Type":"ContainerStarted","Data":"462aff14f99fa761ce3f77109bc3749bd344fe88bd7407469a6409bee07ac64b"}
Nov 04 02:12:13.073172 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:13.073132    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-52dc4" event={"ID":"46cceba3-d979-4401-b4f3-decd198016e8","Type":"ContainerStarted","Data":"68ce16ad25b463b0123befdd04838ae8b49e7d4e3c7eaaeffa3a311869e713b0"}
Nov 04 02:12:13.073595 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:13.073264    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-75bd89fdfc-52dc4"
Nov 04 02:12:13.192331 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:13.192286    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/model-serving-api-75bd89fdfc-52dc4" podStartSLOduration=1.610374192 podStartE2EDuration="4.192272758s" totalImagesPullingTime="2.581898566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:12:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:12:13.191416707 +0000 UTC m=+369.117639341" watchObservedRunningTime="2026-11-04 02:12:13.192272758 +0000 UTC m=+369.118495391"
Nov 04 02:12:14.021307 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:14.021241    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:14.080545 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:14.080505    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-jp7lh" event={"ID":"32f36ce2-d4d6-4b82-9ffe-ff5796462611","Type":"ContainerStarted","Data":"b6c5bc12b11a4aaa7f0e3dfe45365e2cebcc8cacb2719342f33872c2ba277fc7"}
Nov 04 02:12:14.080950 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:14.080720    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-776cc66d47-jp7lh"
Nov 04 02:12:14.200716 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:14.200660    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-776cc66d47-jp7lh" podStartSLOduration=2.82930386 podStartE2EDuration="6.200642294s" totalImagesPullingTime="3.371338434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:12:14.197854813 +0000 UTC m=+370.124077476" watchObservedRunningTime="2026-11-04 02:12:14.200642294 +0000 UTC m=+370.126864928"
Nov 04 02:12:14.205634 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:14.205610    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:15.584687 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:15.584654    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:15.754648 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:15.754607    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:15.935463 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:15.935388    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:16.113379 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:16.113348    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:16.294605 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:16.294569    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:16.494988 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:16.494957    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:16.695231 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:16.695160    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:16.867381 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:16.867355    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:24.087055 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:24.087024    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-75bd89fdfc-52dc4"
Nov 04 02:12:30.855101 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:30.855069    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:31.077155 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:31.077110    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:31.170859 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:31.170790    4969 generic.go:356] "Generic (PLEG): container finished" podID="6d4bc677-6239-47b2-92c1-1924e9cd1b54" containerID="538f64ee0995a627da78a3bc69f644e9a8c8c29932931154e19223d2ce055269" exitCode=0
Nov 04 02:12:31.170977 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:31.170864    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg" event={"ID":"6d4bc677-6239-47b2-92c1-1924e9cd1b54","Type":"ContainerDied","Data":"538f64ee0995a627da78a3bc69f644e9a8c8c29932931154e19223d2ce055269"}
Nov 04 02:12:31.171195 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:31.171179    4969 scope.go:122] "RemoveContainer" containerID="538f64ee0995a627da78a3bc69f644e9a8c8c29932931154e19223d2ce055269"
Nov 04 02:12:31.172189 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:31.172159    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:12:31.172283 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:31.172243    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:12:31.172345 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:31.172295    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:12:31.405480 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:31.397963    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-55445df7f8-pvcfj"]
Nov 04 02:12:31.406324 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:31.406297    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-55445df7f8-pvcfj"
Nov 04 02:12:31.422987 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:31.422928    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2e18c33-58c3-48a3-8c25-bd1adbcbfa59-openshift-service-ca\") pod \"maas-api-55445df7f8-pvcfj\" (UID: \"e2e18c33-58c3-48a3-8c25-bd1adbcbfa59\") " pod="odh-ai-gateway-infra/maas-api-55445df7f8-pvcfj"
Nov 04 02:12:31.422987 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:31.422964    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxsqv\" (UniqueName: \"kubernetes.io/projected/e2e18c33-58c3-48a3-8c25-bd1adbcbfa59-kube-api-access-jxsqv\") pod \"maas-api-55445df7f8-pvcfj\" (UID: \"e2e18c33-58c3-48a3-8c25-bd1adbcbfa59\") " pod="odh-ai-gateway-infra/maas-api-55445df7f8-pvcfj"
Nov 04 02:12:31.423193 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:31.423038    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2e18c33-58c3-48a3-8c25-bd1adbcbfa59-metrics-certs\") pod \"maas-api-55445df7f8-pvcfj\" (UID: \"e2e18c33-58c3-48a3-8c25-bd1adbcbfa59\") " pod="odh-ai-gateway-infra/maas-api-55445df7f8-pvcfj"
Nov 04 02:12:31.424922 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:31.423304    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2e18c33-58c3-48a3-8c25-bd1adbcbfa59-maas-api-tls\") pod \"maas-api-55445df7f8-pvcfj\" (UID: \"e2e18c33-58c3-48a3-8c25-bd1adbcbfa59\") " pod="odh-ai-gateway-infra/maas-api-55445df7f8-pvcfj"
Nov 04 02:12:31.457801 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:31.457765    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-55445df7f8-pvcfj"]
Nov 04 02:12:31.459027 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:31.459000    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 04 02:12:31.459177 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:31.459129    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-p956t\""
Nov 04 02:12:31.459236 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:31.459201    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Nov 04 02:12:31.459295 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:31.459260    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 02:12:31.459884 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:31.459860    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 02:12:31.524429 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:31.524392    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2e18c33-58c3-48a3-8c25-bd1adbcbfa59-maas-api-tls\") pod \"maas-api-55445df7f8-pvcfj\" (UID: \"e2e18c33-58c3-48a3-8c25-bd1adbcbfa59\") " pod="odh-ai-gateway-infra/maas-api-55445df7f8-pvcfj"
Nov 04 02:12:31.524590 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:31.524441    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2e18c33-58c3-48a3-8c25-bd1adbcbfa59-openshift-service-ca\") pod \"maas-api-55445df7f8-pvcfj\" (UID: \"e2e18c33-58c3-48a3-8c25-bd1adbcbfa59\") " pod="odh-ai-gateway-infra/maas-api-55445df7f8-pvcfj"
Nov 04 02:12:31.524590 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:31.524464    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxsqv\" (UniqueName: \"kubernetes.io/projected/e2e18c33-58c3-48a3-8c25-bd1adbcbfa59-kube-api-access-jxsqv\") pod \"maas-api-55445df7f8-pvcfj\" (UID: \"e2e18c33-58c3-48a3-8c25-bd1adbcbfa59\") " pod="odh-ai-gateway-infra/maas-api-55445df7f8-pvcfj"
Nov 04 02:12:31.524590 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:31.524543    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2e18c33-58c3-48a3-8c25-bd1adbcbfa59-metrics-certs\") pod \"maas-api-55445df7f8-pvcfj\" (UID: \"e2e18c33-58c3-48a3-8c25-bd1adbcbfa59\") " pod="odh-ai-gateway-infra/maas-api-55445df7f8-pvcfj"
Nov 04 02:12:31.525281 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:31.525256    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2e18c33-58c3-48a3-8c25-bd1adbcbfa59-openshift-service-ca\") pod \"maas-api-55445df7f8-pvcfj\" (UID: \"e2e18c33-58c3-48a3-8c25-bd1adbcbfa59\") " pod="odh-ai-gateway-infra/maas-api-55445df7f8-pvcfj"
Nov 04 02:12:31.527201 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:31.527179    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2e18c33-58c3-48a3-8c25-bd1adbcbfa59-maas-api-tls\") pod \"maas-api-55445df7f8-pvcfj\" (UID: \"e2e18c33-58c3-48a3-8c25-bd1adbcbfa59\") " pod="odh-ai-gateway-infra/maas-api-55445df7f8-pvcfj"
Nov 04 02:12:31.527440 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:31.527420    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2e18c33-58c3-48a3-8c25-bd1adbcbfa59-metrics-certs\") pod \"maas-api-55445df7f8-pvcfj\" (UID: \"e2e18c33-58c3-48a3-8c25-bd1adbcbfa59\") " pod="odh-ai-gateway-infra/maas-api-55445df7f8-pvcfj"
Nov 04 02:12:31.588876 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:31.588834    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxsqv\" (UniqueName: \"kubernetes.io/projected/e2e18c33-58c3-48a3-8c25-bd1adbcbfa59-kube-api-access-jxsqv\") pod \"maas-api-55445df7f8-pvcfj\" (UID: \"e2e18c33-58c3-48a3-8c25-bd1adbcbfa59\") " pod="odh-ai-gateway-infra/maas-api-55445df7f8-pvcfj"
Nov 04 02:12:31.723118 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:31.723017    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-55445df7f8-pvcfj"
Nov 04 02:12:32.121551 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:32.121522    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-55445df7f8-pvcfj"]
Nov 04 02:12:32.123493 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:12:32.123465    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2e18c33_58c3_48a3_8c25_bd1adbcbfa59.slice/crio-1d7f6e3ff349072f926d83f19f9ec3ac29b73ef43982dfe91435a736b372bd8a WatchSource:0}: Error finding container 1d7f6e3ff349072f926d83f19f9ec3ac29b73ef43982dfe91435a736b372bd8a: Status 404 returned error can't find the container with id 1d7f6e3ff349072f926d83f19f9ec3ac29b73ef43982dfe91435a736b372bd8a
Nov 04 02:12:32.186523 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:32.186486    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-55445df7f8-pvcfj" event={"ID":"e2e18c33-58c3-48a3-8c25-bd1adbcbfa59","Type":"ContainerStarted","Data":"1d7f6e3ff349072f926d83f19f9ec3ac29b73ef43982dfe91435a736b372bd8a"}
Nov 04 02:12:32.188449 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:32.188422    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg" event={"ID":"6d4bc677-6239-47b2-92c1-1924e9cd1b54","Type":"ContainerStarted","Data":"b355121d719e954f4d895c110bb2d81fdfe82262dab8edb74f9e9dbaec1fc438"}
Nov 04 02:12:32.854737 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:32.854704    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:12:32.855040 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:32.855017    4969 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-8bglg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Nov 04 02:12:32.855166 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:32.855070    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg" podUID="6d4bc677-6239-47b2-92c1-1924e9cd1b54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Nov 04 02:12:33.854956 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:33.854922    4969 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-8bglg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Nov 04 02:12:33.855386 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:33.854984    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg" podUID="6d4bc677-6239-47b2-92c1-1924e9cd1b54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Nov 04 02:12:34.293942 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:12:34.293837    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:12:34.854873 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:34.854833    4969 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-8bglg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Nov 04 02:12:34.855047 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:34.854895    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg" podUID="6d4bc677-6239-47b2-92c1-1924e9cd1b54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Nov 04 02:12:35.207298 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:35.207213    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-55445df7f8-pvcfj" event={"ID":"e2e18c33-58c3-48a3-8c25-bd1adbcbfa59","Type":"ContainerStarted","Data":"a8c9f5ccbb8b49160cea303c14e416653eb3f056a4d4dc4f17e037dc16e8b052"}
Nov 04 02:12:35.207455 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:35.207415    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-55445df7f8-pvcfj"
Nov 04 02:12:35.330651 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:35.330595    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-55445df7f8-pvcfj" podStartSLOduration=2.029148714 podStartE2EDuration="4.330581467s" totalImagesPullingTime="2.301432753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:12:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:12:35.329232982 +0000 UTC m=+391.255455615" watchObservedRunningTime="2026-11-04 02:12:35.330581467 +0000 UTC m=+391.256804137"
Nov 04 02:12:35.855012 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:35.854975    4969 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-8bglg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Nov 04 02:12:35.855208 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:35.855036    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg" podUID="6d4bc677-6239-47b2-92c1-1924e9cd1b54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Nov 04 02:12:36.854822 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:36.854788    4969 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-8bglg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Nov 04 02:12:36.854983 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:36.854846    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg" podUID="6d4bc677-6239-47b2-92c1-1924e9cd1b54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Nov 04 02:12:37.027122 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:37.027083    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:37.195461 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:37.195376    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:37.376983 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:37.376942    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:12:37.858610 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:37.858579    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:12:37.858894 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:37.858873    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:12:37.859681 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:37.859663    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-8bglg"
Nov 04 02:12:41.217483 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:41.217455    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-55445df7f8-pvcfj"
Nov 04 02:12:45.090400 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:45.090325    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-776cc66d47-jp7lh"
Nov 04 02:12:56.156524 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:12:56.156492    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:02.003163 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:13:02.003130    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:02.180058 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:13:02.180026    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:04.260830 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:13:04.260794    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:13:34.260318 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:13:34.260281    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:13:50.000921 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:13:50.000881    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:50.183752 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:13:50.183719    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:56.514486 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:13:56.514450    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:13:56.866082 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:13:56.866002    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:04.259485 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:14:04.259446    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:14:09.255644 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:09.255603    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:11.667951 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:11.667920    4969 generic.go:356] "Generic (PLEG): container finished" podID="1bedd7c8-c5a9-419d-90bc-de1428f21eb3" containerID="43c0c7055af355bb804c6f27e25132a49b904d4dcd49213f5f3bc245c81f533c" exitCode=1
Nov 04 02:14:11.668284 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:11.668004    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f759c744c-cr7gd" event={"ID":"1bedd7c8-c5a9-419d-90bc-de1428f21eb3","Type":"ContainerDied","Data":"43c0c7055af355bb804c6f27e25132a49b904d4dcd49213f5f3bc245c81f533c"}
Nov 04 02:14:11.668358 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:11.668342    4969 scope.go:122] "RemoveContainer" containerID="43c0c7055af355bb804c6f27e25132a49b904d4dcd49213f5f3bc245c81f533c"
Nov 04 02:14:12.674292 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:12.674264    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f759c744c-cr7gd" event={"ID":"1bedd7c8-c5a9-419d-90bc-de1428f21eb3","Type":"ContainerStarted","Data":"b8b1b8e3e608850d39b65c66f71459bad1897768875ae6e26498214f8e0bc2e8"}
Nov 04 02:14:12.674594 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:12.674511    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-f759c744c-cr7gd"
Nov 04 02:14:12.992611 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:12.992517    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm"]
Nov 04 02:14:12.996837 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:12.996817    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm"
Nov 04 02:14:13.052316 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:13.052288    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Nov 04 02:14:13.052516 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:13.052313    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Nov 04 02:14:13.052640 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:13.052322    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Nov 04 02:14:13.052640 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:13.052383    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6f7xg\""
Nov 04 02:14:13.054224 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:13.054204    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm"]
Nov 04 02:14:13.120610 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:13.120582    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:13.124670 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:13.124649    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/759da135-51be-4cc8-bd82-a32afb1f8456-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm\" (UID: \"759da135-51be-4cc8-bd82-a32afb1f8456\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm"
Nov 04 02:14:13.124779 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:13.124691    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/759da135-51be-4cc8-bd82-a32afb1f8456-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm\" (UID: \"759da135-51be-4cc8-bd82-a32afb1f8456\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm"
Nov 04 02:14:13.124779 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:13.124760    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/759da135-51be-4cc8-bd82-a32afb1f8456-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm\" (UID: \"759da135-51be-4cc8-bd82-a32afb1f8456\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm"
Nov 04 02:14:13.124863 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:13.124786    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/759da135-51be-4cc8-bd82-a32afb1f8456-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm\" (UID: \"759da135-51be-4cc8-bd82-a32afb1f8456\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm"
Nov 04 02:14:13.124863 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:13.124810    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/759da135-51be-4cc8-bd82-a32afb1f8456-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm\" (UID: \"759da135-51be-4cc8-bd82-a32afb1f8456\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm"
Nov 04 02:14:13.124941 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:13.124919    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxq6g\" (UniqueName: \"kubernetes.io/projected/759da135-51be-4cc8-bd82-a32afb1f8456-kube-api-access-sxq6g\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm\" (UID: \"759da135-51be-4cc8-bd82-a32afb1f8456\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm"
Nov 04 02:14:13.226082 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:13.226009    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/759da135-51be-4cc8-bd82-a32afb1f8456-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm\" (UID: \"759da135-51be-4cc8-bd82-a32afb1f8456\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm"
Nov 04 02:14:13.226082 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:13.226057    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/759da135-51be-4cc8-bd82-a32afb1f8456-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm\" (UID: \"759da135-51be-4cc8-bd82-a32afb1f8456\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm"
Nov 04 02:14:13.226309 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:13.226117    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/759da135-51be-4cc8-bd82-a32afb1f8456-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm\" (UID: \"759da135-51be-4cc8-bd82-a32afb1f8456\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm"
Nov 04 02:14:13.226309 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:13.226143    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/759da135-51be-4cc8-bd82-a32afb1f8456-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm\" (UID: \"759da135-51be-4cc8-bd82-a32afb1f8456\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm"
Nov 04 02:14:13.226309 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:13.226170    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxq6g\" (UniqueName: \"kubernetes.io/projected/759da135-51be-4cc8-bd82-a32afb1f8456-kube-api-access-sxq6g\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm\" (UID: \"759da135-51be-4cc8-bd82-a32afb1f8456\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm"
Nov 04 02:14:13.226309 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:13.226245    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/759da135-51be-4cc8-bd82-a32afb1f8456-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm\" (UID: \"759da135-51be-4cc8-bd82-a32afb1f8456\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm"
Nov 04 02:14:13.226557 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:13.226529    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/759da135-51be-4cc8-bd82-a32afb1f8456-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm\" (UID: \"759da135-51be-4cc8-bd82-a32afb1f8456\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm"
Nov 04 02:14:13.226672 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:13.226555    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/759da135-51be-4cc8-bd82-a32afb1f8456-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm\" (UID: \"759da135-51be-4cc8-bd82-a32afb1f8456\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm"
Nov 04 02:14:13.226672 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:13.226654    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/759da135-51be-4cc8-bd82-a32afb1f8456-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm\" (UID: \"759da135-51be-4cc8-bd82-a32afb1f8456\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm"
Nov 04 02:14:13.228870 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:13.228846    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/759da135-51be-4cc8-bd82-a32afb1f8456-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm\" (UID: \"759da135-51be-4cc8-bd82-a32afb1f8456\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm"
Nov 04 02:14:13.229148 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:13.229130    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/759da135-51be-4cc8-bd82-a32afb1f8456-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm\" (UID: \"759da135-51be-4cc8-bd82-a32afb1f8456\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm"
Nov 04 02:14:13.286762 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:13.286740    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxq6g\" (UniqueName: \"kubernetes.io/projected/759da135-51be-4cc8-bd82-a32afb1f8456-kube-api-access-sxq6g\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm\" (UID: \"759da135-51be-4cc8-bd82-a32afb1f8456\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm"
Nov 04 02:14:13.314537 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:13.314511    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm"
Nov 04 02:14:13.688115 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:13.688028    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm"]
Nov 04 02:14:13.691163 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:14:13.691137    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod759da135_51be_4cc8_bd82_a32afb1f8456.slice/crio-6282569c7700298b8507f953d1544daa8ce5af08122ed40d1a017a62b0e322d3 WatchSource:0}: Error finding container 6282569c7700298b8507f953d1544daa8ce5af08122ed40d1a017a62b0e322d3: Status 404 returned error can't find the container with id 6282569c7700298b8507f953d1544daa8ce5af08122ed40d1a017a62b0e322d3
Nov 04 02:14:14.689040 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:14.688978    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm" event={"ID":"759da135-51be-4cc8-bd82-a32afb1f8456","Type":"ContainerStarted","Data":"6282569c7700298b8507f953d1544daa8ce5af08122ed40d1a017a62b0e322d3"}
Nov 04 02:14:15.696623 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:15.696525    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm" event={"ID":"759da135-51be-4cc8-bd82-a32afb1f8456","Type":"ContainerStarted","Data":"05324695756e4b8c53cc1f91d8a77a0299dfbcdf2b6439bd250eceb81ceed9c8"}
Nov 04 02:14:15.816066 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:15.816009    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm" podStartSLOduration=2.169956814 podStartE2EDuration="3.815997826s" totalImagesPullingTime="1.646041012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:14:15.814259499 +0000 UTC m=+491.740482133" watchObservedRunningTime="2026-11-04 02:14:15.815997826 +0000 UTC m=+491.742220459"
Nov 04 02:14:16.315197 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:16.315166    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm"
Nov 04 02:14:17.296337 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:17.296302    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:20.627118 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:20.627073    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq"]
Nov 04 02:14:20.630660 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:20.630640    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq"
Nov 04 02:14:20.712369 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:20.712278    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Nov 04 02:14:20.737961 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:20.737926    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq"]
Nov 04 02:14:20.753592 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:20.753562    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:20.799917 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:20.799884    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c591d3b-6aba-4492-8409-54a8c36d1b85-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq\" (UID: \"8c591d3b-6aba-4492-8409-54a8c36d1b85\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq"
Nov 04 02:14:20.800040 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:20.799955    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c591d3b-6aba-4492-8409-54a8c36d1b85-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq\" (UID: \"8c591d3b-6aba-4492-8409-54a8c36d1b85\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq"
Nov 04 02:14:20.800111 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:20.800049    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c591d3b-6aba-4492-8409-54a8c36d1b85-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq\" (UID: \"8c591d3b-6aba-4492-8409-54a8c36d1b85\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq"
Nov 04 02:14:20.800186 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:20.800166    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c591d3b-6aba-4492-8409-54a8c36d1b85-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq\" (UID: \"8c591d3b-6aba-4492-8409-54a8c36d1b85\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq"
Nov 04 02:14:20.800225 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:20.800207    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz9pd\" (UniqueName: \"kubernetes.io/projected/8c591d3b-6aba-4492-8409-54a8c36d1b85-kube-api-access-bz9pd\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq\" (UID: \"8c591d3b-6aba-4492-8409-54a8c36d1b85\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq"
Nov 04 02:14:20.800349 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:20.800332    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c591d3b-6aba-4492-8409-54a8c36d1b85-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq\" (UID: \"8c591d3b-6aba-4492-8409-54a8c36d1b85\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq"
Nov 04 02:14:20.901332 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:20.901267    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c591d3b-6aba-4492-8409-54a8c36d1b85-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq\" (UID: \"8c591d3b-6aba-4492-8409-54a8c36d1b85\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq"
Nov 04 02:14:20.901332 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:20.901310    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c591d3b-6aba-4492-8409-54a8c36d1b85-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq\" (UID: \"8c591d3b-6aba-4492-8409-54a8c36d1b85\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq"
Nov 04 02:14:20.901643 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:20.901347    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c591d3b-6aba-4492-8409-54a8c36d1b85-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq\" (UID: \"8c591d3b-6aba-4492-8409-54a8c36d1b85\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq"
Nov 04 02:14:20.901643 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:20.901369    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c591d3b-6aba-4492-8409-54a8c36d1b85-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq\" (UID: \"8c591d3b-6aba-4492-8409-54a8c36d1b85\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq"
Nov 04 02:14:20.901643 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:20.901391    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c591d3b-6aba-4492-8409-54a8c36d1b85-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq\" (UID: \"8c591d3b-6aba-4492-8409-54a8c36d1b85\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq"
Nov 04 02:14:20.901643 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:20.901413    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz9pd\" (UniqueName: \"kubernetes.io/projected/8c591d3b-6aba-4492-8409-54a8c36d1b85-kube-api-access-bz9pd\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq\" (UID: \"8c591d3b-6aba-4492-8409-54a8c36d1b85\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq"
Nov 04 02:14:20.901871 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:20.901701    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c591d3b-6aba-4492-8409-54a8c36d1b85-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq\" (UID: \"8c591d3b-6aba-4492-8409-54a8c36d1b85\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq"
Nov 04 02:14:20.901871 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:20.901791    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c591d3b-6aba-4492-8409-54a8c36d1b85-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq\" (UID: \"8c591d3b-6aba-4492-8409-54a8c36d1b85\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq"
Nov 04 02:14:20.901964 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:20.901869    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c591d3b-6aba-4492-8409-54a8c36d1b85-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq\" (UID: \"8c591d3b-6aba-4492-8409-54a8c36d1b85\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq"
Nov 04 02:14:20.903715 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:20.903688    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c591d3b-6aba-4492-8409-54a8c36d1b85-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq\" (UID: \"8c591d3b-6aba-4492-8409-54a8c36d1b85\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq"
Nov 04 02:14:20.903822 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:20.903807    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c591d3b-6aba-4492-8409-54a8c36d1b85-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq\" (UID: \"8c591d3b-6aba-4492-8409-54a8c36d1b85\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq"
Nov 04 02:14:20.961873 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:20.961851    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz9pd\" (UniqueName: \"kubernetes.io/projected/8c591d3b-6aba-4492-8409-54a8c36d1b85-kube-api-access-bz9pd\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq\" (UID: \"8c591d3b-6aba-4492-8409-54a8c36d1b85\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq"
Nov 04 02:14:21.242671 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:21.242600    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq"
Nov 04 02:14:21.612996 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:21.612972    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq"]
Nov 04 02:14:21.615010 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:14:21.614983    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c591d3b_6aba_4492_8409_54a8c36d1b85.slice/crio-7672ddc2033cbeda95870609819851585626ee4daf0fd5887ab5d39815d48102 WatchSource:0}: Error finding container 7672ddc2033cbeda95870609819851585626ee4daf0fd5887ab5d39815d48102: Status 404 returned error can't find the container with id 7672ddc2033cbeda95870609819851585626ee4daf0fd5887ab5d39815d48102
Nov 04 02:14:21.717192 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:21.717166    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:21.729943 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:21.729917    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq" event={"ID":"8c591d3b-6aba-4492-8409-54a8c36d1b85","Type":"ContainerStarted","Data":"7672ddc2033cbeda95870609819851585626ee4daf0fd5887ab5d39815d48102"}
Nov 04 02:14:22.736620 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:22.736588    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq" event={"ID":"8c591d3b-6aba-4492-8409-54a8c36d1b85","Type":"ContainerStarted","Data":"bc82ff386829b5b5d97c59cc04641901eea52eac840561dab4fbe145c38c5460"}
Nov 04 02:14:22.854670 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:22.854623    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq" podStartSLOduration=2.283822698 podStartE2EDuration="2.85460986s" totalImagesPullingTime="570.787162ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:14:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:14:22.852334646 +0000 UTC m=+498.778557279" watchObservedRunningTime="2026-11-04 02:14:22.85460986 +0000 UTC m=+498.780832492"
Nov 04 02:14:23.243450 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:23.243420    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq"
Nov 04 02:14:23.315685 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:23.315658    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm"
Nov 04 02:14:23.328511 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:23.328491    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm"
Nov 04 02:14:23.681473 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:23.681445    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-f759c744c-cr7gd"
Nov 04 02:14:23.752585 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:23.752559    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm"
Nov 04 02:14:24.385410 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:24.385376    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:27.998815 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:27.998718    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:31.243192 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:31.243161    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq"
Nov 04 02:14:31.255906 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:31.255885    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq"
Nov 04 02:14:31.793134 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:31.793081    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq"
Nov 04 02:14:31.960721 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:31.960686    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m"]
Nov 04 02:14:31.964336 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:31.964313    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m"
Nov 04 02:14:32.016275 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:32.016252    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Nov 04 02:14:32.024073 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:32.024032    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m"]
Nov 04 02:14:32.089710 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:32.089636    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:32.093805 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:32.093779    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0efb04e0-fa93-4504-9a40-c48f5451d9cd-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6m4m\" (UID: \"0efb04e0-fa93-4504-9a40-c48f5451d9cd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m"
Nov 04 02:14:32.093910 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:32.093812    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0efb04e0-fa93-4504-9a40-c48f5451d9cd-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6m4m\" (UID: \"0efb04e0-fa93-4504-9a40-c48f5451d9cd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m"
Nov 04 02:14:32.093910 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:32.093828    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0efb04e0-fa93-4504-9a40-c48f5451d9cd-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6m4m\" (UID: \"0efb04e0-fa93-4504-9a40-c48f5451d9cd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m"
Nov 04 02:14:32.093910 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:32.093848    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb4dc\" (UniqueName: \"kubernetes.io/projected/0efb04e0-fa93-4504-9a40-c48f5451d9cd-kube-api-access-bb4dc\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6m4m\" (UID: \"0efb04e0-fa93-4504-9a40-c48f5451d9cd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m"
Nov 04 02:14:32.094074 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:32.094028    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0efb04e0-fa93-4504-9a40-c48f5451d9cd-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6m4m\" (UID: \"0efb04e0-fa93-4504-9a40-c48f5451d9cd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m"
Nov 04 02:14:32.094148 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:32.094080    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0efb04e0-fa93-4504-9a40-c48f5451d9cd-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6m4m\" (UID: \"0efb04e0-fa93-4504-9a40-c48f5451d9cd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m"
Nov 04 02:14:32.195394 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:32.195369    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0efb04e0-fa93-4504-9a40-c48f5451d9cd-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6m4m\" (UID: \"0efb04e0-fa93-4504-9a40-c48f5451d9cd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m"
Nov 04 02:14:32.195521 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:32.195403    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0efb04e0-fa93-4504-9a40-c48f5451d9cd-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6m4m\" (UID: \"0efb04e0-fa93-4504-9a40-c48f5451d9cd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m"
Nov 04 02:14:32.195521 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:32.195421    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0efb04e0-fa93-4504-9a40-c48f5451d9cd-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6m4m\" (UID: \"0efb04e0-fa93-4504-9a40-c48f5451d9cd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m"
Nov 04 02:14:32.195521 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:32.195442    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb4dc\" (UniqueName: \"kubernetes.io/projected/0efb04e0-fa93-4504-9a40-c48f5451d9cd-kube-api-access-bb4dc\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6m4m\" (UID: \"0efb04e0-fa93-4504-9a40-c48f5451d9cd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m"
Nov 04 02:14:32.195521 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:32.195510    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0efb04e0-fa93-4504-9a40-c48f5451d9cd-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6m4m\" (UID: \"0efb04e0-fa93-4504-9a40-c48f5451d9cd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m"
Nov 04 02:14:32.195776 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:32.195552    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0efb04e0-fa93-4504-9a40-c48f5451d9cd-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6m4m\" (UID: \"0efb04e0-fa93-4504-9a40-c48f5451d9cd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m"
Nov 04 02:14:32.195868 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:32.195831    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0efb04e0-fa93-4504-9a40-c48f5451d9cd-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6m4m\" (UID: \"0efb04e0-fa93-4504-9a40-c48f5451d9cd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m"
Nov 04 02:14:32.195940 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:32.195896    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0efb04e0-fa93-4504-9a40-c48f5451d9cd-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6m4m\" (UID: \"0efb04e0-fa93-4504-9a40-c48f5451d9cd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m"
Nov 04 02:14:32.195988 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:32.195969    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0efb04e0-fa93-4504-9a40-c48f5451d9cd-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6m4m\" (UID: \"0efb04e0-fa93-4504-9a40-c48f5451d9cd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m"
Nov 04 02:14:32.197821 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:32.197800    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0efb04e0-fa93-4504-9a40-c48f5451d9cd-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6m4m\" (UID: \"0efb04e0-fa93-4504-9a40-c48f5451d9cd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m"
Nov 04 02:14:32.198077 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:32.198057    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0efb04e0-fa93-4504-9a40-c48f5451d9cd-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6m4m\" (UID: \"0efb04e0-fa93-4504-9a40-c48f5451d9cd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m"
Nov 04 02:14:32.256134 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:32.256114    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb4dc\" (UniqueName: \"kubernetes.io/projected/0efb04e0-fa93-4504-9a40-c48f5451d9cd-kube-api-access-bb4dc\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-p6m4m\" (UID: \"0efb04e0-fa93-4504-9a40-c48f5451d9cd\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m"
Nov 04 02:14:32.276923 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:32.276904    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m"
Nov 04 02:14:32.651079 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:32.651054    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m"]
Nov 04 02:14:32.652847 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:14:32.652818    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0efb04e0_fa93_4504_9a40_c48f5451d9cd.slice/crio-36c5bc2fc17356d3ee8237cb5fed4ebf43cec95d56351b37286e064372445ba7 WatchSource:0}: Error finding container 36c5bc2fc17356d3ee8237cb5fed4ebf43cec95d56351b37286e064372445ba7: Status 404 returned error can't find the container with id 36c5bc2fc17356d3ee8237cb5fed4ebf43cec95d56351b37286e064372445ba7
Nov 04 02:14:32.788048 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:32.788017    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m" event={"ID":"0efb04e0-fa93-4504-9a40-c48f5451d9cd","Type":"ContainerStarted","Data":"36c5bc2fc17356d3ee8237cb5fed4ebf43cec95d56351b37286e064372445ba7"}
Nov 04 02:14:33.794342 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:33.794300    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m" event={"ID":"0efb04e0-fa93-4504-9a40-c48f5451d9cd","Type":"ContainerStarted","Data":"239c1c76c9ffd8c24a6b64e32a12c44d53055e9cfcc98e9451293b63bdbdcd98"}
Nov 04 02:14:33.911788 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:33.911725    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m" podStartSLOduration=2.334485715 podStartE2EDuration="2.91171052s" totalImagesPullingTime="577.224805ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:14:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:14:33.910976649 +0000 UTC m=+509.837199286" watchObservedRunningTime="2026-11-04 02:14:33.91171052 +0000 UTC m=+509.837933153"
Nov 04 02:14:34.261581 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:14:34.261543    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:14:34.277079 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:34.277044    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m"
Nov 04 02:14:39.380763 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:39.380730    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:42.277835 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:42.277794    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m"
Nov 04 02:14:42.290831 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:42.290807    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m"
Nov 04 02:14:42.851727 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:42.851695    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-p6m4m"
Nov 04 02:14:56.868502 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:56.868465    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:57.693585 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:57.693547    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58bcb58bd7-lpkwf"]
Nov 04 02:14:57.697497 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:57.697475    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58bcb58bd7-lpkwf"
Nov 04 02:14:57.801512 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:57.801482    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-5nx9g\""
Nov 04 02:14:57.801672 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:57.801528    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Nov 04 02:14:57.804944 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:57.804918    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58bcb58bd7-lpkwf"]
Nov 04 02:14:57.817226 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:57.817203    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4p8t\" (UniqueName: \"kubernetes.io/projected/9627a478-94ef-41ca-b965-3982e66f502f-kube-api-access-l4p8t\") pod \"authorino-58bcb58bd7-lpkwf\" (UID: \"9627a478-94ef-41ca-b965-3982e66f502f\") " pod="kuadrant-system/authorino-58bcb58bd7-lpkwf"
Nov 04 02:14:57.817394 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:57.817373    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9627a478-94ef-41ca-b965-3982e66f502f-tls-cert\") pod \"authorino-58bcb58bd7-lpkwf\" (UID: \"9627a478-94ef-41ca-b965-3982e66f502f\") " pod="kuadrant-system/authorino-58bcb58bd7-lpkwf"
Nov 04 02:14:57.863308 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:57.863283    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-69464886d7-7lf7z"]
Nov 04 02:14:57.867015 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:57.866997    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-69464886d7-7lf7z"
Nov 04 02:14:57.876841 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:57.876819    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:57.918271 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:57.918241    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9627a478-94ef-41ca-b965-3982e66f502f-tls-cert\") pod \"authorino-58bcb58bd7-lpkwf\" (UID: \"9627a478-94ef-41ca-b965-3982e66f502f\") " pod="kuadrant-system/authorino-58bcb58bd7-lpkwf"
Nov 04 02:14:57.918392 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:57.918291    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4p8t\" (UniqueName: \"kubernetes.io/projected/9627a478-94ef-41ca-b965-3982e66f502f-kube-api-access-l4p8t\") pod \"authorino-58bcb58bd7-lpkwf\" (UID: \"9627a478-94ef-41ca-b965-3982e66f502f\") " pod="kuadrant-system/authorino-58bcb58bd7-lpkwf"
Nov 04 02:14:57.918392 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:57.918346    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7c5eb4b-7ff4-4098-bb67-44ed721fa178-metrics-certs\") pod \"ai-gateway-controller-69464886d7-7lf7z\" (UID: \"c7c5eb4b-7ff4-4098-bb67-44ed721fa178\") " pod="opendatahub/ai-gateway-controller-69464886d7-7lf7z"
Nov 04 02:14:57.918392 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:57.918368    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h745l\" (UniqueName: \"kubernetes.io/projected/c7c5eb4b-7ff4-4098-bb67-44ed721fa178-kube-api-access-h745l\") pod \"ai-gateway-controller-69464886d7-7lf7z\" (UID: \"c7c5eb4b-7ff4-4098-bb67-44ed721fa178\") " pod="opendatahub/ai-gateway-controller-69464886d7-7lf7z"
Nov 04 02:14:57.918908 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:57.918880    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Nov 04 02:14:57.919013 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:57.918944    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Nov 04 02:14:57.919013 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:57.918957    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-7gb5b\""
Nov 04 02:14:57.921372 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:57.921351    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9627a478-94ef-41ca-b965-3982e66f502f-tls-cert\") pod \"authorino-58bcb58bd7-lpkwf\" (UID: \"9627a478-94ef-41ca-b965-3982e66f502f\") " pod="kuadrant-system/authorino-58bcb58bd7-lpkwf"
Nov 04 02:14:57.925867 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:57.925842    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-69464886d7-7lf7z"]
Nov 04 02:14:57.978767 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:57.978711    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4p8t\" (UniqueName: \"kubernetes.io/projected/9627a478-94ef-41ca-b965-3982e66f502f-kube-api-access-l4p8t\") pod \"authorino-58bcb58bd7-lpkwf\" (UID: \"9627a478-94ef-41ca-b965-3982e66f502f\") " pod="kuadrant-system/authorino-58bcb58bd7-lpkwf"
Nov 04 02:14:58.012070 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:58.012050    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58bcb58bd7-lpkwf"
Nov 04 02:14:58.019195 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:58.019173    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7c5eb4b-7ff4-4098-bb67-44ed721fa178-metrics-certs\") pod \"ai-gateway-controller-69464886d7-7lf7z\" (UID: \"c7c5eb4b-7ff4-4098-bb67-44ed721fa178\") " pod="opendatahub/ai-gateway-controller-69464886d7-7lf7z"
Nov 04 02:14:58.019292 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:58.019202    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h745l\" (UniqueName: \"kubernetes.io/projected/c7c5eb4b-7ff4-4098-bb67-44ed721fa178-kube-api-access-h745l\") pod \"ai-gateway-controller-69464886d7-7lf7z\" (UID: \"c7c5eb4b-7ff4-4098-bb67-44ed721fa178\") " pod="opendatahub/ai-gateway-controller-69464886d7-7lf7z"
Nov 04 02:14:58.021401 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:58.021380    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7c5eb4b-7ff4-4098-bb67-44ed721fa178-metrics-certs\") pod \"ai-gateway-controller-69464886d7-7lf7z\" (UID: \"c7c5eb4b-7ff4-4098-bb67-44ed721fa178\") " pod="opendatahub/ai-gateway-controller-69464886d7-7lf7z"
Nov 04 02:14:58.080021 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:58.079988    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h745l\" (UniqueName: \"kubernetes.io/projected/c7c5eb4b-7ff4-4098-bb67-44ed721fa178-kube-api-access-h745l\") pod \"ai-gateway-controller-69464886d7-7lf7z\" (UID: \"c7c5eb4b-7ff4-4098-bb67-44ed721fa178\") " pod="opendatahub/ai-gateway-controller-69464886d7-7lf7z"
Nov 04 02:14:58.087030 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:58.087007    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:14:58.180088 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:58.180061    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-69464886d7-7lf7z"
Nov 04 02:14:58.385559 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:58.385538    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58bcb58bd7-lpkwf"]
Nov 04 02:14:58.386766 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:14:58.386736    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9627a478_94ef_41ca_b965_3982e66f502f.slice/crio-b5af49287ab1232e4e3077eeaed24274f951b88c175da9d4287c145c43158823 WatchSource:0}: Error finding container b5af49287ab1232e4e3077eeaed24274f951b88c175da9d4287c145c43158823: Status 404 returned error can't find the container with id b5af49287ab1232e4e3077eeaed24274f951b88c175da9d4287c145c43158823
Nov 04 02:14:58.552111 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:58.552074    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-69464886d7-7lf7z"]
Nov 04 02:14:58.553758 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:14:58.553735    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7c5eb4b_7ff4_4098_bb67_44ed721fa178.slice/crio-a70ebd000b489f5eb4a7decd8724f94be08cbd32d847d6db2a50d98fc85fbae5 WatchSource:0}: Error finding container a70ebd000b489f5eb4a7decd8724f94be08cbd32d847d6db2a50d98fc85fbae5: Status 404 returned error can't find the container with id a70ebd000b489f5eb4a7decd8724f94be08cbd32d847d6db2a50d98fc85fbae5
Nov 04 02:14:58.929586 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:58.929492    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58bcb58bd7-lpkwf" event={"ID":"9627a478-94ef-41ca-b965-3982e66f502f","Type":"ContainerStarted","Data":"b5af49287ab1232e4e3077eeaed24274f951b88c175da9d4287c145c43158823"}
Nov 04 02:14:58.930727 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:14:58.930700    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-69464886d7-7lf7z" event={"ID":"c7c5eb4b-7ff4-4098-bb67-44ed721fa178","Type":"ContainerStarted","Data":"a70ebd000b489f5eb4a7decd8724f94be08cbd32d847d6db2a50d98fc85fbae5"}
Nov 04 02:15:01.947219 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:01.947127    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58bcb58bd7-lpkwf" event={"ID":"9627a478-94ef-41ca-b965-3982e66f502f","Type":"ContainerStarted","Data":"98f32a0be9bf0edac9f765cd4e35bc85b1aaab28948fe194532bc0bb4fbe0143"}
Nov 04 02:15:02.062184 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:02.062130    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-58bcb58bd7-lpkwf" podStartSLOduration=1.834487365 podStartE2EDuration="5.062117027s" totalImagesPullingTime="3.227629662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:14:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:15:02.060497811 +0000 UTC m=+537.986720444" watchObservedRunningTime="2026-11-04 02:15:02.062117027 +0000 UTC m=+537.988339657"
Nov 04 02:15:04.260492 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:15:04.260456    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:15:06.208116 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.208057    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"]
Nov 04 02:15:06.212534 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.212512    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.260137 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.256455    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"]
Nov 04 02:15:06.261689 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.261656    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.303495 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.303460    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-628mm\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.303649 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.303505    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-628mm\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.303649 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.303531    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-628mm\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.303649 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.303588    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-628mm\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.303781 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.303691    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-628mm\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.303781 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.303712    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-628mm\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.303781 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.303745    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-628mm\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.303904 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.303826    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzw94\" (UniqueName: \"kubernetes.io/projected/162cba19-31aa-4cf1-80d1-96ef638c8ee7-kube-api-access-fzw94\") pod \"maas-default-gateway-openshift-default-6cc458d887-628mm\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.303943 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.303923    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-628mm\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.317769 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.317730    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"]
Nov 04 02:15:06.370416 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.370213    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"]
Nov 04 02:15:06.404619 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.404588    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-qnjkb\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.404797 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.404632    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-628mm\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.404797 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.404709    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-628mm\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.404797 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.404750    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-628mm\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.404797 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.404786    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-628mm\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.405018 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.404823    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzw94\" (UniqueName: \"kubernetes.io/projected/162cba19-31aa-4cf1-80d1-96ef638c8ee7-kube-api-access-fzw94\") pod \"maas-default-gateway-openshift-default-6cc458d887-628mm\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.405018 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.404852    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-qnjkb\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.405018 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.404880    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-628mm\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.405018 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.404931    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-qnjkb\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.405018 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.404958    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-qnjkb\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.405018 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.405007    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-qnjkb\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.405360 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.405036    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-qnjkb\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.405360 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.405078    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-628mm\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.405360 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.405133    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-qnjkb\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.405360 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.405166    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-628mm\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.405360 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.405198    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-qnjkb\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.405360 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.405239    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-628mm\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.405360 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.405274    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmk67\" (UniqueName: \"kubernetes.io/projected/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-kube-api-access-xmk67\") pod \"maas-default-gateway-openshift-default-6cc458d887-qnjkb\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.405771 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.405745    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-628mm\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.405912 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.405836    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-628mm\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.406026 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.406008    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-628mm\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.406216 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.406195    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-628mm\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.406546 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.406527    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-628mm\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.408519 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.408472    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-628mm\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.408655 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.408546    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-628mm\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.465900 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.465833    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-628mm\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.466074 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.466047    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzw94\" (UniqueName: \"kubernetes.io/projected/162cba19-31aa-4cf1-80d1-96ef638c8ee7-kube-api-access-fzw94\") pod \"maas-default-gateway-openshift-default-6cc458d887-628mm\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.506820 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.506784    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-qnjkb\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.506990 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.506838    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-qnjkb\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.506990 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.506880    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmk67\" (UniqueName: \"kubernetes.io/projected/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-kube-api-access-xmk67\") pod \"maas-default-gateway-openshift-default-6cc458d887-qnjkb\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.506990 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.506919    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-qnjkb\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.506990 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.506976    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-qnjkb\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.507269 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.507019    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-qnjkb\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.507269 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.507047    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-qnjkb\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.507269 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.507105    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-qnjkb\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.507269 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.507137    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-qnjkb\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.507481 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.507310    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-credential-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-qnjkb\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.507481 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.507339    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-workload-socket\") pod \"maas-default-gateway-openshift-default-6cc458d887-qnjkb\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.507609 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.507539    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-workload-certs\") pod \"maas-default-gateway-openshift-default-6cc458d887-qnjkb\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.507667 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.507635    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istio-data\") pod \"maas-default-gateway-openshift-default-6cc458d887-qnjkb\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.507739 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.507721    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6cc458d887-qnjkb\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.509518 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.509493    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istio-envoy\") pod \"maas-default-gateway-openshift-default-6cc458d887-qnjkb\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.509889 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.509871    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6cc458d887-qnjkb\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.530201 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.530176    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:06.571110 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.568524    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmk67\" (UniqueName: \"kubernetes.io/projected/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-kube-api-access-xmk67\") pod \"maas-default-gateway-openshift-default-6cc458d887-qnjkb\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.571110 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.568720    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istio-token\") pod \"maas-default-gateway-openshift-default-6cc458d887-qnjkb\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.582126 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.578565    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:06.856059 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.856025    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Nov 04 02:15:06.961013 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.960983    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"]
Nov 04 02:15:06.964715 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:15:06.964681    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod162cba19_31aa_4cf1_80d1_96ef638c8ee7.slice/crio-a2d8fdd5a828769e887ab3ad8d4861aa547cafe24091329596d30548c7eea9a1 WatchSource:0}: Error finding container a2d8fdd5a828769e887ab3ad8d4861aa547cafe24091329596d30548c7eea9a1: Status 404 returned error can't find the container with id a2d8fdd5a828769e887ab3ad8d4861aa547cafe24091329596d30548c7eea9a1
Nov 04 02:15:06.967024 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.966996    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:15:06.967110 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.967060    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:15:06.967110 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.967087    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:15:06.989483 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.989454    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm" event={"ID":"162cba19-31aa-4cf1-80d1-96ef638c8ee7","Type":"ContainerStarted","Data":"a2d8fdd5a828769e887ab3ad8d4861aa547cafe24091329596d30548c7eea9a1"}
Nov 04 02:15:06.990984 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.990956    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-69464886d7-7lf7z" event={"ID":"c7c5eb4b-7ff4-4098-bb67-44ed721fa178","Type":"ContainerStarted","Data":"ad0083bdf78fcfae6e627420e5b864b517e3fb51b0d9a3e9bbe9d9545dec359b"}
Nov 04 02:15:06.991130 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:06.991112    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-69464886d7-7lf7z"
Nov 04 02:15:07.062382 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:07.062301    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"]
Nov 04 02:15:07.066284 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:15:07.066258    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0093557b_c9ae_4bd8_b9ea_019f62cd69d1.slice/crio-2bec1e6b44e8d7308fb430374ce081edab503ab90b6cba91b7e9df51a3f0c998 WatchSource:0}: Error finding container 2bec1e6b44e8d7308fb430374ce081edab503ab90b6cba91b7e9df51a3f0c998: Status 404 returned error can't find the container with id 2bec1e6b44e8d7308fb430374ce081edab503ab90b6cba91b7e9df51a3f0c998
Nov 04 02:15:07.068542 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:07.068515    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:15:07.068618 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:07.068578    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:15:07.068618 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:07.068606    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:15:07.109167 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:07.109110    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-69464886d7-7lf7z" podStartSLOduration=1.8599413889999998 podStartE2EDuration="10.109078472s" totalImagesPullingTime="8.249137083s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:14:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:15:07.105771489 +0000 UTC m=+543.031994123" watchObservedRunningTime="2026-11-04 02:15:07.109078472 +0000 UTC m=+543.035301107"
Nov 04 02:15:08.017751 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:08.017712    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb" event={"ID":"0093557b-c9ae-4bd8-b9ea-019f62cd69d1","Type":"ContainerStarted","Data":"de716654f44b991a9176f2acd678e85223b08068263740534ab1e06fa10c3dab"}
Nov 04 02:15:08.018686 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:08.018525    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb" event={"ID":"0093557b-c9ae-4bd8-b9ea-019f62cd69d1","Type":"ContainerStarted","Data":"2bec1e6b44e8d7308fb430374ce081edab503ab90b6cba91b7e9df51a3f0c998"}
Nov 04 02:15:08.021195 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:08.021173    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm" event={"ID":"162cba19-31aa-4cf1-80d1-96ef638c8ee7","Type":"ContainerStarted","Data":"a5d04015b9da1b3245491fdfbda7ae7fc34473a7bf6b814cdfccf66d2ff5bdb2"}
Nov 04 02:15:08.151354 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:08.151297    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb" podStartSLOduration=2.151280544 podStartE2EDuration="2.151280544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:15:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:15:08.149875337 +0000 UTC m=+544.076097970" watchObservedRunningTime="2026-11-04 02:15:08.151280544 +0000 UTC m=+544.077503176"
Nov 04 02:15:08.307170 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:08.306107    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm" podStartSLOduration=2.306057124 podStartE2EDuration="2.306057124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:15:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:15:08.287344906 +0000 UTC m=+544.213567539" watchObservedRunningTime="2026-11-04 02:15:08.306057124 +0000 UTC m=+544.232279758"
Nov 04 02:15:08.530610 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:08.530573    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:08.532381 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:08.532357    4969 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-628mm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Nov 04 02:15:08.532567 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:08.532548    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm" podUID="162cba19-31aa-4cf1-80d1-96ef638c8ee7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Nov 04 02:15:08.582556 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:08.582428    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:08.584988 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:08.584870    4969 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-qnjkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 04 02:15:08.584988 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:08.584926    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb" podUID="0093557b-c9ae-4bd8-b9ea-019f62cd69d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 04 02:15:09.258504 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:09.258469    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:09.440349 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:09.440316    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:09.530707 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:09.530622    4969 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-628mm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Nov 04 02:15:09.530707 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:09.530683    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm" podUID="162cba19-31aa-4cf1-80d1-96ef638c8ee7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Nov 04 02:15:09.598117 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:09.587029    4969 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-qnjkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 04 02:15:09.598117 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:09.587111    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb" podUID="0093557b-c9ae-4bd8-b9ea-019f62cd69d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 04 02:15:10.530931 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:10.530896    4969 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-628mm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Nov 04 02:15:10.531362 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:10.530947    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm" podUID="162cba19-31aa-4cf1-80d1-96ef638c8ee7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Nov 04 02:15:10.581114 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:10.580427    4969 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-qnjkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 04 02:15:10.581114 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:10.580490    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb" podUID="0093557b-c9ae-4bd8-b9ea-019f62cd69d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 04 02:15:10.997171 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:10.989158    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:11.198116 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:11.197633    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:11.377115 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:11.369009    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-7vwj2"]
Nov 04 02:15:11.399117 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:11.392821    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-7vwj2"
Nov 04 02:15:11.430641 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:11.430609    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-7vwj2"]
Nov 04 02:15:11.481116 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:11.477675    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Nov 04 02:15:11.481116 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:11.477864    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-q6kgs\""
Nov 04 02:15:11.520676 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:11.520650    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7bb9489676-pl7h2"]
Nov 04 02:15:11.521034 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:11.521007    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20254f43-5d9a-4d1a-9b14-6fe6580dd397-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-7vwj2\" (UID: \"20254f43-5d9a-4d1a-9b14-6fe6580dd397\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-7vwj2"
Nov 04 02:15:11.521184 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:11.521159    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpzg7\" (UniqueName: \"kubernetes.io/projected/20254f43-5d9a-4d1a-9b14-6fe6580dd397-kube-api-access-vpzg7\") pod \"payload-pre-processing-7b98bb678c-7vwj2\" (UID: \"20254f43-5d9a-4d1a-9b14-6fe6580dd397\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-7vwj2"
Nov 04 02:15:11.524716 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:11.524699    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bb9489676-pl7h2"
Nov 04 02:15:11.530626 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:11.530599    4969 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-628mm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Nov 04 02:15:11.530716 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:11.530650    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm" podUID="162cba19-31aa-4cf1-80d1-96ef638c8ee7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Nov 04 02:15:11.555292 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:11.555263    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bb9489676-pl7h2"]
Nov 04 02:15:11.607117 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:11.606720    4969 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-qnjkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 04 02:15:11.607117 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:11.606776    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb" podUID="0093557b-c9ae-4bd8-b9ea-019f62cd69d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 04 02:15:11.623670 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:11.623628    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20254f43-5d9a-4d1a-9b14-6fe6580dd397-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-7vwj2\" (UID: \"20254f43-5d9a-4d1a-9b14-6fe6580dd397\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-7vwj2"
Nov 04 02:15:11.623802 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:11.623760    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpzg7\" (UniqueName: \"kubernetes.io/projected/20254f43-5d9a-4d1a-9b14-6fe6580dd397-kube-api-access-vpzg7\") pod \"payload-pre-processing-7b98bb678c-7vwj2\" (UID: \"20254f43-5d9a-4d1a-9b14-6fe6580dd397\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-7vwj2"
Nov 04 02:15:11.623851 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:11.623806    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90527deb-17f0-45d8-846a-99add8a3a440-plugins-config-volume\") pod \"payload-processing-7bb9489676-pl7h2\" (UID: \"90527deb-17f0-45d8-846a-99add8a3a440\") " pod="openshift-ingress/payload-processing-7bb9489676-pl7h2"
Nov 04 02:15:11.623851 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:11.623840    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f6hn\" (UniqueName: \"kubernetes.io/projected/90527deb-17f0-45d8-846a-99add8a3a440-kube-api-access-7f6hn\") pod \"payload-processing-7bb9489676-pl7h2\" (UID: \"90527deb-17f0-45d8-846a-99add8a3a440\") " pod="openshift-ingress/payload-processing-7bb9489676-pl7h2"
Nov 04 02:15:11.624891 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:11.624867    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20254f43-5d9a-4d1a-9b14-6fe6580dd397-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-7vwj2\" (UID: \"20254f43-5d9a-4d1a-9b14-6fe6580dd397\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-7vwj2"
Nov 04 02:15:11.710764 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:11.710265    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpzg7\" (UniqueName: \"kubernetes.io/projected/20254f43-5d9a-4d1a-9b14-6fe6580dd397-kube-api-access-vpzg7\") pod \"payload-pre-processing-7b98bb678c-7vwj2\" (UID: \"20254f43-5d9a-4d1a-9b14-6fe6580dd397\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-7vwj2"
Nov 04 02:15:11.722912 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:11.722886    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-7vwj2"
Nov 04 02:15:11.724446 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:11.724419    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90527deb-17f0-45d8-846a-99add8a3a440-plugins-config-volume\") pod \"payload-processing-7bb9489676-pl7h2\" (UID: \"90527deb-17f0-45d8-846a-99add8a3a440\") " pod="openshift-ingress/payload-processing-7bb9489676-pl7h2"
Nov 04 02:15:11.724521 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:11.724469    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f6hn\" (UniqueName: \"kubernetes.io/projected/90527deb-17f0-45d8-846a-99add8a3a440-kube-api-access-7f6hn\") pod \"payload-processing-7bb9489676-pl7h2\" (UID: \"90527deb-17f0-45d8-846a-99add8a3a440\") " pod="openshift-ingress/payload-processing-7bb9489676-pl7h2"
Nov 04 02:15:11.725541 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:11.725519    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90527deb-17f0-45d8-846a-99add8a3a440-plugins-config-volume\") pod \"payload-processing-7bb9489676-pl7h2\" (UID: \"90527deb-17f0-45d8-846a-99add8a3a440\") " pod="openshift-ingress/payload-processing-7bb9489676-pl7h2"
Nov 04 02:15:11.803706 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:11.803670    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f6hn\" (UniqueName: \"kubernetes.io/projected/90527deb-17f0-45d8-846a-99add8a3a440-kube-api-access-7f6hn\") pod \"payload-processing-7bb9489676-pl7h2\" (UID: \"90527deb-17f0-45d8-846a-99add8a3a440\") " pod="openshift-ingress/payload-processing-7bb9489676-pl7h2"
Nov 04 02:15:11.838692 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:11.838247    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bb9489676-pl7h2"
Nov 04 02:15:12.208668 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:12.192896    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-7vwj2"]
Nov 04 02:15:12.342448 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:12.342423    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bb9489676-pl7h2"]
Nov 04 02:15:12.344255 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:15:12.344228    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90527deb_17f0_45d8_846a_99add8a3a440.slice/crio-b203d3c4cb4bb32ece5268af5db466ce65893cc418b472692ff3b71a028826aa WatchSource:0}: Error finding container b203d3c4cb4bb32ece5268af5db466ce65893cc418b472692ff3b71a028826aa: Status 404 returned error can't find the container with id b203d3c4cb4bb32ece5268af5db466ce65893cc418b472692ff3b71a028826aa
Nov 04 02:15:12.531185 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:12.531146    4969 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-628mm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Nov 04 02:15:12.531357 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:12.531201    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm" podUID="162cba19-31aa-4cf1-80d1-96ef638c8ee7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Nov 04 02:15:12.584195 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:12.583760    4969 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-qnjkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 04 02:15:12.584195 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:12.583822    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb" podUID="0093557b-c9ae-4bd8-b9ea-019f62cd69d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 04 02:15:13.054805 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:13.054768    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-7vwj2" event={"ID":"20254f43-5d9a-4d1a-9b14-6fe6580dd397","Type":"ContainerStarted","Data":"4805d429da24f169d5ff0bec8c03167bd58a1583f11ff1548e2cc7a1637a8e4a"}
Nov 04 02:15:13.056022 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:13.055990    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bb9489676-pl7h2" event={"ID":"90527deb-17f0-45d8-846a-99add8a3a440","Type":"ContainerStarted","Data":"b203d3c4cb4bb32ece5268af5db466ce65893cc418b472692ff3b71a028826aa"}
Nov 04 02:15:13.531482 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:13.531449    4969 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-628mm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Nov 04 02:15:13.531641 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:13.531507    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm" podUID="162cba19-31aa-4cf1-80d1-96ef638c8ee7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Nov 04 02:15:13.579599 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:13.579553    4969 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-qnjkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 04 02:15:13.579733 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:13.579632    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb" podUID="0093557b-c9ae-4bd8-b9ea-019f62cd69d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 04 02:15:14.531238 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:14.531199    4969 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-628mm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Nov 04 02:15:14.531731 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:14.531264    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm" podUID="162cba19-31aa-4cf1-80d1-96ef638c8ee7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Nov 04 02:15:14.579073 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:14.579039    4969 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-qnjkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 04 02:15:14.579267 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:14.579140    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb" podUID="0093557b-c9ae-4bd8-b9ea-019f62cd69d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 04 02:15:16.082787 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:16.082754    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bb9489676-pl7h2" event={"ID":"90527deb-17f0-45d8-846a-99add8a3a440","Type":"ContainerStarted","Data":"82e303e926a3ed1908cad405ce5b32336b32d0b01a23cc62c76751e1ab3d488e"}
Nov 04 02:15:16.083201 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:16.082942    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bb9489676-pl7h2"
Nov 04 02:15:16.201824 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:16.201748    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7bb9489676-pl7h2" podStartSLOduration=1.840025402 podStartE2EDuration="5.201727266s" totalImagesPullingTime="3.361701864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:15:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:15:16.200892261 +0000 UTC m=+552.127114895" watchObservedRunningTime="2026-11-04 02:15:16.201727266 +0000 UTC m=+552.127949901"
Nov 04 02:15:16.532715 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:16.532674    4969 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-628mm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 02:15:16.532886 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:16.532743    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm" podUID="162cba19-31aa-4cf1-80d1-96ef638c8ee7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:15:16.598149 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:16.598114    4969 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-qnjkb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 02:15:16.598321 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:16.598167    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb" podUID="0093557b-c9ae-4bd8-b9ea-019f62cd69d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:15:16.645692 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:16.645659    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:16.646047 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:16.646029    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:16.646812 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:16.646793    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:15:17.090484 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:17.090445    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-7vwj2" event={"ID":"20254f43-5d9a-4d1a-9b14-6fe6580dd397","Type":"ContainerStarted","Data":"83f7580a0deef261d26b9485a1783ed256d77501c62f82c4734fea0ee4d2d0f0"}
Nov 04 02:15:17.206166 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:17.206121    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7b98bb678c-7vwj2" podStartSLOduration=2.362190916 podStartE2EDuration="6.20610699s" totalImagesPullingTime="3.843916074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:15:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:15:17.205809269 +0000 UTC m=+553.132031913" watchObservedRunningTime="2026-11-04 02:15:17.20610699 +0000 UTC m=+553.132329614"
Nov 04 02:15:17.574303 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:17.574250    4969 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-628mm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 02:15:17.574485 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:17.574342    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm" podUID="162cba19-31aa-4cf1-80d1-96ef638c8ee7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:15:18.023551 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:18.023515    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-69464886d7-7lf7z"
Nov 04 02:15:18.120486 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:18.120462    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-7vwj2"
Nov 04 02:15:18.366606 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:18.366514    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-7cf899df5-cdbnv"]
Nov 04 02:15:18.370976 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:18.370956    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7cf899df5-cdbnv"
Nov 04 02:15:18.421590 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:18.421559    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6x8b\" (UniqueName: \"kubernetes.io/projected/31d608ea-2e0c-4bfc-b80a-370eb6a7ad01-kube-api-access-c6x8b\") pod \"ai-gateway-controller-7cf899df5-cdbnv\" (UID: \"31d608ea-2e0c-4bfc-b80a-370eb6a7ad01\") " pod="opendatahub/ai-gateway-controller-7cf899df5-cdbnv"
Nov 04 02:15:18.421774 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:18.421649    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31d608ea-2e0c-4bfc-b80a-370eb6a7ad01-metrics-certs\") pod \"ai-gateway-controller-7cf899df5-cdbnv\" (UID: \"31d608ea-2e0c-4bfc-b80a-370eb6a7ad01\") " pod="opendatahub/ai-gateway-controller-7cf899df5-cdbnv"
Nov 04 02:15:18.428885 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:18.428857    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7cf899df5-cdbnv"]
Nov 04 02:15:18.522975 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:18.522944    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6x8b\" (UniqueName: \"kubernetes.io/projected/31d608ea-2e0c-4bfc-b80a-370eb6a7ad01-kube-api-access-c6x8b\") pod \"ai-gateway-controller-7cf899df5-cdbnv\" (UID: \"31d608ea-2e0c-4bfc-b80a-370eb6a7ad01\") " pod="opendatahub/ai-gateway-controller-7cf899df5-cdbnv"
Nov 04 02:15:18.523182 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:18.523030    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31d608ea-2e0c-4bfc-b80a-370eb6a7ad01-metrics-certs\") pod \"ai-gateway-controller-7cf899df5-cdbnv\" (UID: \"31d608ea-2e0c-4bfc-b80a-370eb6a7ad01\") " pod="opendatahub/ai-gateway-controller-7cf899df5-cdbnv"
Nov 04 02:15:18.525810 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:18.525774    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31d608ea-2e0c-4bfc-b80a-370eb6a7ad01-metrics-certs\") pod \"ai-gateway-controller-7cf899df5-cdbnv\" (UID: \"31d608ea-2e0c-4bfc-b80a-370eb6a7ad01\") " pod="opendatahub/ai-gateway-controller-7cf899df5-cdbnv"
Nov 04 02:15:18.615819 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:18.615754    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6x8b\" (UniqueName: \"kubernetes.io/projected/31d608ea-2e0c-4bfc-b80a-370eb6a7ad01-kube-api-access-c6x8b\") pod \"ai-gateway-controller-7cf899df5-cdbnv\" (UID: \"31d608ea-2e0c-4bfc-b80a-370eb6a7ad01\") " pod="opendatahub/ai-gateway-controller-7cf899df5-cdbnv"
Nov 04 02:15:18.616878 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:18.616219    4969 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6cc458d887-628mm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 02:15:18.616878 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:18.616268    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm" podUID="162cba19-31aa-4cf1-80d1-96ef638c8ee7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:15:18.685884 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:18.685850    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7cf899df5-cdbnv"
Nov 04 02:15:19.077160 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:19.077127    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7cf899df5-cdbnv"]
Nov 04 02:15:19.128628 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:19.128593    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7cf899df5-cdbnv" event={"ID":"31d608ea-2e0c-4bfc-b80a-370eb6a7ad01","Type":"ContainerStarted","Data":"85caf342ecade9c31f60a12bea4ff00a46d7d2f6db3dc9f4d0a91d8bfe119db0"}
Nov 04 02:15:19.212773 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:19.212740    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:19.213363 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:19.213335    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:19.214284 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:19.214255    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:15:20.134765 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:20.134731    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7cf899df5-cdbnv" event={"ID":"31d608ea-2e0c-4bfc-b80a-370eb6a7ad01","Type":"ContainerStarted","Data":"d2a77a0b8a51811a17cbeb2f9e8df606ebe7feee5cc0a8fe7e6b5069d2b6e7c0"}
Nov 04 02:15:20.250149 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:20.250023    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-7cf899df5-cdbnv" podStartSLOduration=1.3824756 podStartE2EDuration="2.250005685s" totalImagesPullingTime="867.530085ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:15:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:15:20.249389054 +0000 UTC m=+556.175611687" watchObservedRunningTime="2026-11-04 02:15:20.250005685 +0000 UTC m=+556.176228319"
Nov 04 02:15:21.140551 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:21.140525    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-7cf899df5-cdbnv"
Nov 04 02:15:27.094114 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:27.094074    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7bb9489676-pl7h2"
Nov 04 02:15:29.131626 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:29.131594    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-7vwj2"
Nov 04 02:15:29.851123 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:29.851080    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:32.148205 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:32.148177    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-7cf899df5-cdbnv"
Nov 04 02:15:32.315782 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:32.315751    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-69464886d7-7lf7z"]
Nov 04 02:15:32.315982 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:32.315945    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-69464886d7-7lf7z" podUID="c7c5eb4b-7ff4-4098-bb67-44ed721fa178" containerName="manager" containerID="cri-o://ad0083bdf78fcfae6e627420e5b864b517e3fb51b0d9a3e9bbe9d9545dec359b" gracePeriod=30
Nov 04 02:15:32.566922 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:32.566897    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-69464886d7-7lf7z"
Nov 04 02:15:32.644949 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:32.644922    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c7c5eb4b-7ff4-4098-bb67-44ed721fa178-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7c5eb4b-7ff4-4098-bb67-44ed721fa178-metrics-certs\") pod \"c7c5eb4b-7ff4-4098-bb67-44ed721fa178\" (UID: \"c7c5eb4b-7ff4-4098-bb67-44ed721fa178\") "
Nov 04 02:15:32.645071 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:32.644982    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c7c5eb4b-7ff4-4098-bb67-44ed721fa178-kube-api-access-h745l\" (UniqueName: \"kubernetes.io/projected/c7c5eb4b-7ff4-4098-bb67-44ed721fa178-kube-api-access-h745l\") pod \"c7c5eb4b-7ff4-4098-bb67-44ed721fa178\" (UID: \"c7c5eb4b-7ff4-4098-bb67-44ed721fa178\") "
Nov 04 02:15:32.646954 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:32.646917    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7c5eb4b-7ff4-4098-bb67-44ed721fa178-metrics-certs" pod "c7c5eb4b-7ff4-4098-bb67-44ed721fa178" (UID: "c7c5eb4b-7ff4-4098-bb67-44ed721fa178"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:15:32.647050 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:32.646982    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7c5eb4b-7ff4-4098-bb67-44ed721fa178-kube-api-access-h745l" pod "c7c5eb4b-7ff4-4098-bb67-44ed721fa178" (UID: "c7c5eb4b-7ff4-4098-bb67-44ed721fa178"). InnerVolumeSpecName "kube-api-access-h745l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:15:32.745812 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:32.745736    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h745l\" (UniqueName: \"kubernetes.io/projected/c7c5eb4b-7ff4-4098-bb67-44ed721fa178-kube-api-access-h745l\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:15:32.745812 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:32.745770    4969 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7c5eb4b-7ff4-4098-bb67-44ed721fa178-metrics-certs\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:15:33.206745 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:33.206711    4969 generic.go:356] "Generic (PLEG): container finished" podID="c7c5eb4b-7ff4-4098-bb67-44ed721fa178" containerID="ad0083bdf78fcfae6e627420e5b864b517e3fb51b0d9a3e9bbe9d9545dec359b" exitCode=0
Nov 04 02:15:33.207248 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:33.206763    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-69464886d7-7lf7z"
Nov 04 02:15:33.207248 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:33.206789    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-69464886d7-7lf7z" event={"ID":"c7c5eb4b-7ff4-4098-bb67-44ed721fa178","Type":"ContainerDied","Data":"ad0083bdf78fcfae6e627420e5b864b517e3fb51b0d9a3e9bbe9d9545dec359b"}
Nov 04 02:15:33.207248 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:33.206832    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-69464886d7-7lf7z" event={"ID":"c7c5eb4b-7ff4-4098-bb67-44ed721fa178","Type":"ContainerDied","Data":"a70ebd000b489f5eb4a7decd8724f94be08cbd32d847d6db2a50d98fc85fbae5"}
Nov 04 02:15:33.207248 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:33.206854    4969 scope.go:122] "RemoveContainer" containerID="ad0083bdf78fcfae6e627420e5b864b517e3fb51b0d9a3e9bbe9d9545dec359b"
Nov 04 02:15:33.216735 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:33.216718    4969 scope.go:122] "RemoveContainer" containerID="ad0083bdf78fcfae6e627420e5b864b517e3fb51b0d9a3e9bbe9d9545dec359b"
Nov 04 02:15:33.217012 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:15:33.216995    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad0083bdf78fcfae6e627420e5b864b517e3fb51b0d9a3e9bbe9d9545dec359b\": container with ID starting with ad0083bdf78fcfae6e627420e5b864b517e3fb51b0d9a3e9bbe9d9545dec359b not found: ID does not exist" containerID="ad0083bdf78fcfae6e627420e5b864b517e3fb51b0d9a3e9bbe9d9545dec359b"
Nov 04 02:15:33.217056 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:33.217020    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad0083bdf78fcfae6e627420e5b864b517e3fb51b0d9a3e9bbe9d9545dec359b"} err="failed to get container status \"ad0083bdf78fcfae6e627420e5b864b517e3fb51b0d9a3e9bbe9d9545dec359b\": rpc error: code = NotFound desc = could not find container \"ad0083bdf78fcfae6e627420e5b864b517e3fb51b0d9a3e9bbe9d9545dec359b\": container with ID starting with ad0083bdf78fcfae6e627420e5b864b517e3fb51b0d9a3e9bbe9d9545dec359b not found: ID does not exist"
Nov 04 02:15:33.370784 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:33.370756    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-69464886d7-7lf7z"]
Nov 04 02:15:33.419906 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:33.419881    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-69464886d7-7lf7z"]
Nov 04 02:15:34.109215 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:34.109170    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:34.201347 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:34.201309    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c7c5eb4b-7ff4-4098-bb67-44ed721fa178" path="/var/lib/kubelet/pods/c7c5eb4b-7ff4-4098-bb67-44ed721fa178/volumes"
Nov 04 02:15:34.262248 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:15:34.262215    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:15:34.729963 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:34.729920    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:37.415689 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:37.415638    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:37.593483 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:37.593450    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:39.053310 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:39.053278    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:39.231046 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:39.230993    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:15:57.214642 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:15:57.214610    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:04.255727 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:04.255692    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jcqmp_16f00163-6ec3-4c10-81e4-c8a501e15e82/console-operator/2.log"
Nov 04 02:16:04.260136 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:04.260116    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jcqmp_16f00163-6ec3-4c10-81e4-c8a501e15e82/console-operator/2.log"
Nov 04 02:16:04.269601 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:16:04.269571    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:16:06.354449 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:06.354417    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:06.525366 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:06.525336    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:06.993009 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:06.992973    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:07.220048 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:07.220015    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:07.404086 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:07.404053    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:07.660360 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:07.660289    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:07.834684 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:07.834649    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:08.015974 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:08.015943    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:08.201255 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:08.201217    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:08.386796 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:08.386703    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:08.578962 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:08.578925    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:08.772396 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:08.772357    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:08.948425 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:08.948391    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:09.256682 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:09.256596    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:09.527019 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:09.526990    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:11.765746 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:11.765712    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:11.955773 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:11.955742    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:12.141613 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:12.141528    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:12.577484 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:12.577450    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:12.758985 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:12.758953    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:12.939583 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:12.939504    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:13.235978 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:13.235888    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:13.417360 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:13.417333    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:13.597847 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:13.597813    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:13.875234 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:13.875158    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:14.046269 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:14.046240    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:14.231421 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:14.231337    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:14.689803 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:14.689744    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:14.869166 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:14.869121    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:15.048626 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:15.048584    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:16.433995 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:16.433964    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:16.611299 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:16.611242    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:16.792828 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:16.792798    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:16.994475 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:16.994424    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:17.194867 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:17.194717    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:17.380820 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:17.380786    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc"]
Nov 04 02:16:17.381307 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:17.381294    4969 cpu_manager.go:414] "containerMap: removing container" podUID="c7c5eb4b-7ff4-4098-bb67-44ed721fa178" containerName="manager"
Nov 04 02:16:17.381356 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:17.381308    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c7c5eb4b-7ff4-4098-bb67-44ed721fa178" containerName="manager"
Nov 04 02:16:17.381391 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:17.381380    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="c7c5eb4b-7ff4-4098-bb67-44ed721fa178" containerName="manager"
Nov 04 02:16:17.384814 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:17.384797    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc"
Nov 04 02:16:17.437287 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:17.437263    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-5a27ba\""
Nov 04 02:16:17.459953 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:17.459879    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq45v\" (UniqueName: \"kubernetes.io/projected/85d5905f-412e-4e4b-8553-9adb4a72afea-kube-api-access-vq45v\") pod \"payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc\" (UID: \"85d5905f-412e-4e4b-8553-9adb4a72afea\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc"
Nov 04 02:16:17.460074 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:17.459949    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85d5905f-412e-4e4b-8553-9adb4a72afea-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc\" (UID: \"85d5905f-412e-4e4b-8553-9adb4a72afea\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc"
Nov 04 02:16:17.487018 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:17.486982    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-5a27ba-dockercfg-fwq6j\""
Nov 04 02:16:17.489535 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:17.489509    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc"]
Nov 04 02:16:17.540121 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:17.540070    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl"]
Nov 04 02:16:17.544634 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:17.544610    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl"
Nov 04 02:16:17.560448 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:17.560425    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85d5905f-412e-4e4b-8553-9adb4a72afea-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc\" (UID: \"85d5905f-412e-4e4b-8553-9adb4a72afea\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc"
Nov 04 02:16:17.560552 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:17.560516    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq45v\" (UniqueName: \"kubernetes.io/projected/85d5905f-412e-4e4b-8553-9adb4a72afea-kube-api-access-vq45v\") pod \"payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc\" (UID: \"85d5905f-412e-4e4b-8553-9adb4a72afea\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc"
Nov 04 02:16:17.561036 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:17.561017    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85d5905f-412e-4e4b-8553-9adb4a72afea-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc\" (UID: \"85d5905f-412e-4e4b-8553-9adb4a72afea\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc"
Nov 04 02:16:17.583129 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:17.583086    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl"]
Nov 04 02:16:17.621356 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:17.621330    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq45v\" (UniqueName: \"kubernetes.io/projected/85d5905f-412e-4e4b-8553-9adb4a72afea-kube-api-access-vq45v\") pod \"payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc\" (UID: \"85d5905f-412e-4e4b-8553-9adb4a72afea\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc"
Nov 04 02:16:17.661921 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:17.661890    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s69bl\" (UniqueName: \"kubernetes.io/projected/6fc4dede-0532-4749-8cf2-44d868cd64c0-kube-api-access-s69bl\") pod \"payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl\" (UID: \"6fc4dede-0532-4749-8cf2-44d868cd64c0\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl"
Nov 04 02:16:17.661921 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:17.661923    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fc4dede-0532-4749-8cf2-44d868cd64c0-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl\" (UID: \"6fc4dede-0532-4749-8cf2-44d868cd64c0\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl"
Nov 04 02:16:17.698226 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:17.698198    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc"
Nov 04 02:16:17.764572 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:17.763149    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s69bl\" (UniqueName: \"kubernetes.io/projected/6fc4dede-0532-4749-8cf2-44d868cd64c0-kube-api-access-s69bl\") pod \"payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl\" (UID: \"6fc4dede-0532-4749-8cf2-44d868cd64c0\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl"
Nov 04 02:16:17.764572 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:17.763201    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fc4dede-0532-4749-8cf2-44d868cd64c0-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl\" (UID: \"6fc4dede-0532-4749-8cf2-44d868cd64c0\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl"
Nov 04 02:16:17.764572 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:17.764218    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fc4dede-0532-4749-8cf2-44d868cd64c0-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl\" (UID: \"6fc4dede-0532-4749-8cf2-44d868cd64c0\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl"
Nov 04 02:16:17.824988 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:17.824960    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s69bl\" (UniqueName: \"kubernetes.io/projected/6fc4dede-0532-4749-8cf2-44d868cd64c0-kube-api-access-s69bl\") pod \"payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl\" (UID: \"6fc4dede-0532-4749-8cf2-44d868cd64c0\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl"
Nov 04 02:16:17.858187 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:17.858157    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl"
Nov 04 02:16:18.073155 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:18.073124    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc"]
Nov 04 02:16:18.074705 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:16:18.074676    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85d5905f_412e_4e4b_8553_9adb4a72afea.slice/crio-42e27fdb27378543e398018e821148eec01679b00100046f392aebc77b5a0357 WatchSource:0}: Error finding container 42e27fdb27378543e398018e821148eec01679b00100046f392aebc77b5a0357: Status 404 returned error can't find the container with id 42e27fdb27378543e398018e821148eec01679b00100046f392aebc77b5a0357
Nov 04 02:16:18.230837 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:18.230811    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl"]
Nov 04 02:16:18.232451 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:16:18.232425    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fc4dede_0532_4749_8cf2_44d868cd64c0.slice/crio-e23d79e5fec9b3428523c7de09048a60226727077ded91b11e46c5836ef26321 WatchSource:0}: Error finding container e23d79e5fec9b3428523c7de09048a60226727077ded91b11e46c5836ef26321: Status 404 returned error can't find the container with id e23d79e5fec9b3428523c7de09048a60226727077ded91b11e46c5836ef26321
Nov 04 02:16:18.427928 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:18.427855    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn"]
Nov 04 02:16:18.431697 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:18.431674    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn"
Nov 04 02:16:18.463822 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:18.463786    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc" event={"ID":"85d5905f-412e-4e4b-8553-9adb4a72afea","Type":"ContainerStarted","Data":"450fa8d893d7e7bb5ec085d8b6b11d05e4c2e9bbf7ecb6eea2891070da55e042"}
Nov 04 02:16:18.464214 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:18.463841    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc" event={"ID":"85d5905f-412e-4e4b-8553-9adb4a72afea","Type":"ContainerStarted","Data":"42e27fdb27378543e398018e821148eec01679b00100046f392aebc77b5a0357"}
Nov 04 02:16:18.464214 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:18.463869    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc"
Nov 04 02:16:18.465399 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:18.465378    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl" event={"ID":"6fc4dede-0532-4749-8cf2-44d868cd64c0","Type":"ContainerStarted","Data":"5f69836c0421bf2b5d89d47c02103fdb43828ebdcd749a992b777d527ec913ee"}
Nov 04 02:16:18.465487 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:18.465422    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl" event={"ID":"6fc4dede-0532-4749-8cf2-44d868cd64c0","Type":"ContainerStarted","Data":"e23d79e5fec9b3428523c7de09048a60226727077ded91b11e46c5836ef26321"}
Nov 04 02:16:18.465487 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:18.465443    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl"
Nov 04 02:16:18.469970 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:18.469950    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b999b27-4022-43c6-b758-5e36ef2a1776-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn\" (UID: \"8b999b27-4022-43c6-b758-5e36ef2a1776\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn"
Nov 04 02:16:18.470051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:18.470008    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b999b27-4022-43c6-b758-5e36ef2a1776-maas-api-tls\") pod \"maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn\" (UID: \"8b999b27-4022-43c6-b758-5e36ef2a1776\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn"
Nov 04 02:16:18.470102 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:18.470062    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmccw\" (UniqueName: \"kubernetes.io/projected/8b999b27-4022-43c6-b758-5e36ef2a1776-kube-api-access-lmccw\") pod \"maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn\" (UID: \"8b999b27-4022-43c6-b758-5e36ef2a1776\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn"
Nov 04 02:16:18.470204 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:18.470187    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b999b27-4022-43c6-b758-5e36ef2a1776-metrics-certs\") pod \"maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn\" (UID: \"8b999b27-4022-43c6-b758-5e36ef2a1776\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn"
Nov 04 02:16:18.483467 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:18.483449    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-e34b54-serving-cert\""
Nov 04 02:16:18.490141 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:18.490121    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn"]
Nov 04 02:16:18.571580 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:18.571540    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b999b27-4022-43c6-b758-5e36ef2a1776-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn\" (UID: \"8b999b27-4022-43c6-b758-5e36ef2a1776\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn"
Nov 04 02:16:18.571752 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:18.571706    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b999b27-4022-43c6-b758-5e36ef2a1776-maas-api-tls\") pod \"maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn\" (UID: \"8b999b27-4022-43c6-b758-5e36ef2a1776\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn"
Nov 04 02:16:18.571823 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:18.571763    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmccw\" (UniqueName: \"kubernetes.io/projected/8b999b27-4022-43c6-b758-5e36ef2a1776-kube-api-access-lmccw\") pod \"maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn\" (UID: \"8b999b27-4022-43c6-b758-5e36ef2a1776\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn"
Nov 04 02:16:18.572013 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:18.571832    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b999b27-4022-43c6-b758-5e36ef2a1776-metrics-certs\") pod \"maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn\" (UID: \"8b999b27-4022-43c6-b758-5e36ef2a1776\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn"
Nov 04 02:16:18.572013 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:16:18.572003    4969 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-serving-cert: secret "maas-api-e2e-worker-w0-e34b54-serving-cert" not found
Nov 04 02:16:18.572211 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:16:18.572066    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b999b27-4022-43c6-b758-5e36ef2a1776-maas-api-tls podName:8b999b27-4022-43c6-b758-5e36ef2a1776 nodeName:}" failed. No retries permitted until 2026-11-04 02:16:19.072044683 +0000 UTC m=+614.998267308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8b999b27-4022-43c6-b758-5e36ef2a1776-maas-api-tls") pod "maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn" (UID: "8b999b27-4022-43c6-b758-5e36ef2a1776") : secret "maas-api-e2e-worker-w0-e34b54-serving-cert" not found
Nov 04 02:16:18.572547 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:18.572508    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b999b27-4022-43c6-b758-5e36ef2a1776-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn\" (UID: \"8b999b27-4022-43c6-b758-5e36ef2a1776\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn"
Nov 04 02:16:18.574890 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:18.574869    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b999b27-4022-43c6-b758-5e36ef2a1776-metrics-certs\") pod \"maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn\" (UID: \"8b999b27-4022-43c6-b758-5e36ef2a1776\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn"
Nov 04 02:16:18.633166 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:18.633139    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmccw\" (UniqueName: \"kubernetes.io/projected/8b999b27-4022-43c6-b758-5e36ef2a1776-kube-api-access-lmccw\") pod \"maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn\" (UID: \"8b999b27-4022-43c6-b758-5e36ef2a1776\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn"
Nov 04 02:16:18.654055 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:18.654032    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:18.661433 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:18.661393    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl" podStartSLOduration=1.6613818660000002 podStartE2EDuration="1.661381866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:18.66119275 +0000 UTC m=+614.587415384" watchObservedRunningTime="2026-11-04 02:16:18.661381866 +0000 UTC m=+614.587604499"
Nov 04 02:16:18.776876 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:18.776823    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc" podStartSLOduration=1.776810537 podStartE2EDuration="1.776810537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:18.77545514 +0000 UTC m=+614.701677773" watchObservedRunningTime="2026-11-04 02:16:18.776810537 +0000 UTC m=+614.703033169"
Nov 04 02:16:18.841419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:18.841330    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:19.077289 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:19.077206    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b999b27-4022-43c6-b758-5e36ef2a1776-maas-api-tls\") pod \"maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn\" (UID: \"8b999b27-4022-43c6-b758-5e36ef2a1776\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn"
Nov 04 02:16:19.079677 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:19.079655    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b999b27-4022-43c6-b758-5e36ef2a1776-maas-api-tls\") pod \"maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn\" (UID: \"8b999b27-4022-43c6-b758-5e36ef2a1776\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn"
Nov 04 02:16:19.344831 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:19.344741    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn"
Nov 04 02:16:19.720507 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:19.720478    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn"]
Nov 04 02:16:19.720960 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:16:19.720936    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b999b27_4022_43c6_b758_5e36ef2a1776.slice/crio-7b924b696773ba627c05453ac11616fab9bdec8979467f1c14113704ae40d094 WatchSource:0}: Error finding container 7b924b696773ba627c05453ac11616fab9bdec8979467f1c14113704ae40d094: Status 404 returned error can't find the container with id 7b924b696773ba627c05453ac11616fab9bdec8979467f1c14113704ae40d094
Nov 04 02:16:20.479228 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:20.479195    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn" event={"ID":"8b999b27-4022-43c6-b758-5e36ef2a1776","Type":"ContainerStarted","Data":"7b924b696773ba627c05453ac11616fab9bdec8979467f1c14113704ae40d094"}
Nov 04 02:16:20.587005 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:20.586978    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:20.773888 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:20.773863    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:20.946413 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:20.946372    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:21.486775 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:21.486733    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn" event={"ID":"8b999b27-4022-43c6-b758-5e36ef2a1776","Type":"ContainerStarted","Data":"68462f36be3c50f826bc4ed10709dd7950cf09bde70f42562c2e21ba2b0bdcd5"}
Nov 04 02:16:21.486970 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:21.486802    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn"
Nov 04 02:16:21.603643 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:21.603592    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn" podStartSLOduration=2.740421135 podStartE2EDuration="3.6035659s" totalImagesPullingTime="863.144765ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:21.602883028 +0000 UTC m=+617.529105660" watchObservedRunningTime="2026-11-04 02:16:21.6035659 +0000 UTC m=+617.529788533"
Nov 04 02:16:21.953203 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:21.953144    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:22.139713 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:22.139681    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:22.343864 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:22.343827    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt"]
Nov 04 02:16:22.347686 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:22.347662    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt"
Nov 04 02:16:22.401778 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:22.401718    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-e34b54\""
Nov 04 02:16:22.402038 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:22.401999    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-e34b54-dockercfg-tfz5g\""
Nov 04 02:16:22.405102 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:22.405065    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt"]
Nov 04 02:16:22.405432 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:22.405409    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb250458-8451-45f4-a29b-22d4ccb929c1-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt\" (UID: \"cb250458-8451-45f4-a29b-22d4ccb929c1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt"
Nov 04 02:16:22.405517 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:22.405504    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xjqk\" (UniqueName: \"kubernetes.io/projected/cb250458-8451-45f4-a29b-22d4ccb929c1-kube-api-access-4xjqk\") pod \"payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt\" (UID: \"cb250458-8451-45f4-a29b-22d4ccb929c1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt"
Nov 04 02:16:22.496424 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:22.496390    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx"]
Nov 04 02:16:22.500319 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:22.500297    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx"
Nov 04 02:16:22.506634 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:22.506607    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xjqk\" (UniqueName: \"kubernetes.io/projected/cb250458-8451-45f4-a29b-22d4ccb929c1-kube-api-access-4xjqk\") pod \"payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt\" (UID: \"cb250458-8451-45f4-a29b-22d4ccb929c1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt"
Nov 04 02:16:22.506801 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:22.506782    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb250458-8451-45f4-a29b-22d4ccb929c1-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt\" (UID: \"cb250458-8451-45f4-a29b-22d4ccb929c1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt"
Nov 04 02:16:22.507631 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:22.507611    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb250458-8451-45f4-a29b-22d4ccb929c1-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt\" (UID: \"cb250458-8451-45f4-a29b-22d4ccb929c1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt"
Nov 04 02:16:22.558936 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:22.558902    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx"]
Nov 04 02:16:22.566843 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:22.566820    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xjqk\" (UniqueName: \"kubernetes.io/projected/cb250458-8451-45f4-a29b-22d4ccb929c1-kube-api-access-4xjqk\") pod \"payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt\" (UID: \"cb250458-8451-45f4-a29b-22d4ccb929c1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt"
Nov 04 02:16:22.608242 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:22.608174    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt59x\" (UniqueName: \"kubernetes.io/projected/49e51d64-9c71-4a5c-8ff3-8dd79cc6d487-kube-api-access-gt59x\") pod \"payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx\" (UID: \"49e51d64-9c71-4a5c-8ff3-8dd79cc6d487\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx"
Nov 04 02:16:22.608505 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:22.608485    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49e51d64-9c71-4a5c-8ff3-8dd79cc6d487-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx\" (UID: \"49e51d64-9c71-4a5c-8ff3-8dd79cc6d487\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx"
Nov 04 02:16:22.661585 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:22.661564    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt"
Nov 04 02:16:22.709182 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:22.709150    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49e51d64-9c71-4a5c-8ff3-8dd79cc6d487-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx\" (UID: \"49e51d64-9c71-4a5c-8ff3-8dd79cc6d487\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx"
Nov 04 02:16:22.709319 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:22.709233    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt59x\" (UniqueName: \"kubernetes.io/projected/49e51d64-9c71-4a5c-8ff3-8dd79cc6d487-kube-api-access-gt59x\") pod \"payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx\" (UID: \"49e51d64-9c71-4a5c-8ff3-8dd79cc6d487\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx"
Nov 04 02:16:22.709952 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:22.709931    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49e51d64-9c71-4a5c-8ff3-8dd79cc6d487-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx\" (UID: \"49e51d64-9c71-4a5c-8ff3-8dd79cc6d487\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx"
Nov 04 02:16:22.769862 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:22.769831    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt59x\" (UniqueName: \"kubernetes.io/projected/49e51d64-9c71-4a5c-8ff3-8dd79cc6d487-kube-api-access-gt59x\") pod \"payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx\" (UID: \"49e51d64-9c71-4a5c-8ff3-8dd79cc6d487\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx"
Nov 04 02:16:22.814130 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:22.814075    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx"
Nov 04 02:16:23.036607 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:23.036579    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt"]
Nov 04 02:16:23.038544 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:16:23.038518    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb250458_8451_45f4_a29b_22d4ccb929c1.slice/crio-aa3c31eac0ae68dcedb609a7e0d367b4d058aecf4b36d8b2dc0e183cf8319080 WatchSource:0}: Error finding container aa3c31eac0ae68dcedb609a7e0d367b4d058aecf4b36d8b2dc0e183cf8319080: Status 404 returned error can't find the container with id aa3c31eac0ae68dcedb609a7e0d367b4d058aecf4b36d8b2dc0e183cf8319080
Nov 04 02:16:23.189661 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:23.189634    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx"]
Nov 04 02:16:23.191232 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:16:23.191206    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49e51d64_9c71_4a5c_8ff3_8dd79cc6d487.slice/crio-0589e6df2c2a088d46a0e20daa4536c0a81484d3af7a548b2910d48e2c48afe3 WatchSource:0}: Error finding container 0589e6df2c2a088d46a0e20daa4536c0a81484d3af7a548b2910d48e2c48afe3: Status 404 returned error can't find the container with id 0589e6df2c2a088d46a0e20daa4536c0a81484d3af7a548b2910d48e2c48afe3
Nov 04 02:16:23.505166 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:23.505058    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx" event={"ID":"49e51d64-9c71-4a5c-8ff3-8dd79cc6d487","Type":"ContainerStarted","Data":"c6ed892083f3237b989d6b8d8e48745d437f07bfed053a82aa1c4c95d7f72039"}
Nov 04 02:16:23.505334 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:23.505168    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx" event={"ID":"49e51d64-9c71-4a5c-8ff3-8dd79cc6d487","Type":"ContainerStarted","Data":"0589e6df2c2a088d46a0e20daa4536c0a81484d3af7a548b2910d48e2c48afe3"}
Nov 04 02:16:23.505334 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:23.505201    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx"
Nov 04 02:16:23.506774 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:23.506740    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt" event={"ID":"cb250458-8451-45f4-a29b-22d4ccb929c1","Type":"ContainerStarted","Data":"b5b3c2b18c94a2864db43d71097bda1788e5d6711b6c63414cf905320a69e705"}
Nov 04 02:16:23.506911 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:23.506781    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt" event={"ID":"cb250458-8451-45f4-a29b-22d4ccb929c1","Type":"ContainerStarted","Data":"aa3c31eac0ae68dcedb609a7e0d367b4d058aecf4b36d8b2dc0e183cf8319080"}
Nov 04 02:16:23.506911 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:23.506829    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt"
Nov 04 02:16:23.621756 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:23.621706    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx" podStartSLOduration=1.621692666 podStartE2EDuration="1.621692666s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:23.619533567 +0000 UTC m=+619.545756201" watchObservedRunningTime="2026-11-04 02:16:23.621692666 +0000 UTC m=+619.547915342"
Nov 04 02:16:23.721810 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:23.721775    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv"]
Nov 04 02:16:23.725878 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:23.725856    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv"
Nov 04 02:16:23.737945 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:23.737887    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt" podStartSLOduration=1.737867826 podStartE2EDuration="1.737867826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:23.734486039 +0000 UTC m=+619.660708673" watchObservedRunningTime="2026-11-04 02:16:23.737867826 +0000 UTC m=+619.664090461"
Nov 04 02:16:23.777757 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:23.777728    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w5-dd5d18-serving-cert\""
Nov 04 02:16:23.784794 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:23.784767    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv"]
Nov 04 02:16:23.820362 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:23.820335    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bccedafc-3320-4d2c-9a1f-a38dd01adac2-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv\" (UID: \"bccedafc-3320-4d2c-9a1f-a38dd01adac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv"
Nov 04 02:16:23.820499 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:23.820384    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfqrp\" (UniqueName: \"kubernetes.io/projected/bccedafc-3320-4d2c-9a1f-a38dd01adac2-kube-api-access-cfqrp\") pod \"maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv\" (UID: \"bccedafc-3320-4d2c-9a1f-a38dd01adac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv"
Nov 04 02:16:23.820555 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:23.820519    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bccedafc-3320-4d2c-9a1f-a38dd01adac2-metrics-certs\") pod \"maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv\" (UID: \"bccedafc-3320-4d2c-9a1f-a38dd01adac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv"
Nov 04 02:16:23.820600 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:23.820583    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bccedafc-3320-4d2c-9a1f-a38dd01adac2-maas-api-tls\") pod \"maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv\" (UID: \"bccedafc-3320-4d2c-9a1f-a38dd01adac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv"
Nov 04 02:16:23.921808 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:23.921775    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfqrp\" (UniqueName: \"kubernetes.io/projected/bccedafc-3320-4d2c-9a1f-a38dd01adac2-kube-api-access-cfqrp\") pod \"maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv\" (UID: \"bccedafc-3320-4d2c-9a1f-a38dd01adac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv"
Nov 04 02:16:23.921959 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:23.921823    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bccedafc-3320-4d2c-9a1f-a38dd01adac2-metrics-certs\") pod \"maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv\" (UID: \"bccedafc-3320-4d2c-9a1f-a38dd01adac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv"
Nov 04 02:16:23.921959 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:23.921855    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bccedafc-3320-4d2c-9a1f-a38dd01adac2-maas-api-tls\") pod \"maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv\" (UID: \"bccedafc-3320-4d2c-9a1f-a38dd01adac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv"
Nov 04 02:16:23.921959 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:23.921934    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bccedafc-3320-4d2c-9a1f-a38dd01adac2-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv\" (UID: \"bccedafc-3320-4d2c-9a1f-a38dd01adac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv"
Nov 04 02:16:23.922124 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:16:23.922006    4969 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-serving-cert: secret "maas-api-e2e-worker-w5-dd5d18-serving-cert" not found
Nov 04 02:16:23.922124 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:16:23.922084    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bccedafc-3320-4d2c-9a1f-a38dd01adac2-maas-api-tls podName:bccedafc-3320-4d2c-9a1f-a38dd01adac2 nodeName:}" failed. No retries permitted until 2026-11-04 02:16:24.422062567 +0000 UTC m=+620.348285195 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bccedafc-3320-4d2c-9a1f-a38dd01adac2-maas-api-tls") pod "maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv" (UID: "bccedafc-3320-4d2c-9a1f-a38dd01adac2") : secret "maas-api-e2e-worker-w5-dd5d18-serving-cert" not found
Nov 04 02:16:23.922567 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:23.922544    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bccedafc-3320-4d2c-9a1f-a38dd01adac2-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv\" (UID: \"bccedafc-3320-4d2c-9a1f-a38dd01adac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv"
Nov 04 02:16:23.924198 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:23.924179    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bccedafc-3320-4d2c-9a1f-a38dd01adac2-metrics-certs\") pod \"maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv\" (UID: \"bccedafc-3320-4d2c-9a1f-a38dd01adac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv"
Nov 04 02:16:23.951372 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:23.951350    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:23.982442 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:23.982420    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfqrp\" (UniqueName: \"kubernetes.io/projected/bccedafc-3320-4d2c-9a1f-a38dd01adac2-kube-api-access-cfqrp\") pod \"maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv\" (UID: \"bccedafc-3320-4d2c-9a1f-a38dd01adac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv"
Nov 04 02:16:24.131840 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:24.131761    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:24.427395 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:24.427310    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bccedafc-3320-4d2c-9a1f-a38dd01adac2-maas-api-tls\") pod \"maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv\" (UID: \"bccedafc-3320-4d2c-9a1f-a38dd01adac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv"
Nov 04 02:16:24.429691 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:24.429666    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bccedafc-3320-4d2c-9a1f-a38dd01adac2-maas-api-tls\") pod \"maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv\" (UID: \"bccedafc-3320-4d2c-9a1f-a38dd01adac2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv"
Nov 04 02:16:24.527195 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:24.527169    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:24.643359 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:24.643332    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv"
Nov 04 02:16:24.703323 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:24.703295    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:24.911805 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:24.911772    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk"]
Nov 04 02:16:24.916685 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:24.916668    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk"
Nov 04 02:16:24.970297 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:24.970223    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w5-dd5d18\""
Nov 04 02:16:24.970515 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:24.970358    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w5-dd5d18-dockercfg-dss5h\""
Nov 04 02:16:24.974484 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:24.974458    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk"]
Nov 04 02:16:25.033972 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:25.033937    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62b73809-c52f-4c4a-915d-f03e77df2068-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk\" (UID: \"62b73809-c52f-4c4a-915d-f03e77df2068\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk"
Nov 04 02:16:25.034361 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:25.034328    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8ct6\" (UniqueName: \"kubernetes.io/projected/62b73809-c52f-4c4a-915d-f03e77df2068-kube-api-access-r8ct6\") pod \"payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk\" (UID: \"62b73809-c52f-4c4a-915d-f03e77df2068\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk"
Nov 04 02:16:25.036729 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:25.036709    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv"]
Nov 04 02:16:25.038925 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:16:25.038897    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbccedafc_3320_4d2c_9a1f_a38dd01adac2.slice/crio-bec237e1eb8d8d20558b9ab608cf3a2ff75afda01d1d14cb0936ebb24c46f313 WatchSource:0}: Error finding container bec237e1eb8d8d20558b9ab608cf3a2ff75afda01d1d14cb0936ebb24c46f313: Status 404 returned error can't find the container with id bec237e1eb8d8d20558b9ab608cf3a2ff75afda01d1d14cb0936ebb24c46f313
Nov 04 02:16:25.053374 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:25.053345    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt"]
Nov 04 02:16:25.057300 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:25.057285    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt"
Nov 04 02:16:25.115339 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:25.115314    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt"]
Nov 04 02:16:25.135028 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:25.135003    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e4b23e2-526a-464c-bb05-2f156da5652b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt\" (UID: \"6e4b23e2-526a-464c-bb05-2f156da5652b\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt"
Nov 04 02:16:25.135429 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:25.135053    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8ct6\" (UniqueName: \"kubernetes.io/projected/62b73809-c52f-4c4a-915d-f03e77df2068-kube-api-access-r8ct6\") pod \"payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk\" (UID: \"62b73809-c52f-4c4a-915d-f03e77df2068\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk"
Nov 04 02:16:25.135429 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:25.135190    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7nmf\" (UniqueName: \"kubernetes.io/projected/6e4b23e2-526a-464c-bb05-2f156da5652b-kube-api-access-w7nmf\") pod \"payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt\" (UID: \"6e4b23e2-526a-464c-bb05-2f156da5652b\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt"
Nov 04 02:16:25.135429 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:25.135323    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62b73809-c52f-4c4a-915d-f03e77df2068-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk\" (UID: \"62b73809-c52f-4c4a-915d-f03e77df2068\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk"
Nov 04 02:16:25.136010 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:25.135990    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62b73809-c52f-4c4a-915d-f03e77df2068-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk\" (UID: \"62b73809-c52f-4c4a-915d-f03e77df2068\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk"
Nov 04 02:16:25.198392 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:25.198363    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8ct6\" (UniqueName: \"kubernetes.io/projected/62b73809-c52f-4c4a-915d-f03e77df2068-kube-api-access-r8ct6\") pod \"payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk\" (UID: \"62b73809-c52f-4c4a-915d-f03e77df2068\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk"
Nov 04 02:16:25.232727 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:25.232656    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk"
Nov 04 02:16:25.236021 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:25.235995    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e4b23e2-526a-464c-bb05-2f156da5652b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt\" (UID: \"6e4b23e2-526a-464c-bb05-2f156da5652b\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt"
Nov 04 02:16:25.236137 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:25.236116    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7nmf\" (UniqueName: \"kubernetes.io/projected/6e4b23e2-526a-464c-bb05-2f156da5652b-kube-api-access-w7nmf\") pod \"payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt\" (UID: \"6e4b23e2-526a-464c-bb05-2f156da5652b\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt"
Nov 04 02:16:25.236679 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:25.236656    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e4b23e2-526a-464c-bb05-2f156da5652b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt\" (UID: \"6e4b23e2-526a-464c-bb05-2f156da5652b\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt"
Nov 04 02:16:25.298006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:25.297078    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7nmf\" (UniqueName: \"kubernetes.io/projected/6e4b23e2-526a-464c-bb05-2f156da5652b-kube-api-access-w7nmf\") pod \"payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt\" (UID: \"6e4b23e2-526a-464c-bb05-2f156da5652b\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt"
Nov 04 02:16:25.370808 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:25.370782    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt"
Nov 04 02:16:25.521448 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:25.521400    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv" event={"ID":"bccedafc-3320-4d2c-9a1f-a38dd01adac2","Type":"ContainerStarted","Data":"bec237e1eb8d8d20558b9ab608cf3a2ff75afda01d1d14cb0936ebb24c46f313"}
Nov 04 02:16:25.606372 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:25.606349    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk"]
Nov 04 02:16:25.607776 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:16:25.607748    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62b73809_c52f_4c4a_915d_f03e77df2068.slice/crio-82503543b0a22aa3fea3cb563c50afda8932ac65df6ec8ed99bf04863f4fad3d WatchSource:0}: Error finding container 82503543b0a22aa3fea3cb563c50afda8932ac65df6ec8ed99bf04863f4fad3d: Status 404 returned error can't find the container with id 82503543b0a22aa3fea3cb563c50afda8932ac65df6ec8ed99bf04863f4fad3d
Nov 04 02:16:25.744438 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:25.744411    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt"]
Nov 04 02:16:25.746179 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:16:25.746153    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e4b23e2_526a_464c_bb05_2f156da5652b.slice/crio-9da866380901805a991f6cd741a4ed8b94f2861b68385bc2161d2259e18b60e6 WatchSource:0}: Error finding container 9da866380901805a991f6cd741a4ed8b94f2861b68385bc2161d2259e18b60e6: Status 404 returned error can't find the container with id 9da866380901805a991f6cd741a4ed8b94f2861b68385bc2161d2259e18b60e6
Nov 04 02:16:26.528604 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:26.528570    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv" event={"ID":"bccedafc-3320-4d2c-9a1f-a38dd01adac2","Type":"ContainerStarted","Data":"446063e0152795fdfbbb12d23281e5f54a35598eabb6ea2007e0ad14d90c51ca"}
Nov 04 02:16:26.529075 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:26.528671    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv"
Nov 04 02:16:26.530225 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:26.530199    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt" event={"ID":"6e4b23e2-526a-464c-bb05-2f156da5652b","Type":"ContainerStarted","Data":"0a4f86cf1106335275af3546e544778829c13969a52c56675e077dd5af200500"}
Nov 04 02:16:26.530331 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:26.530247    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt" event={"ID":"6e4b23e2-526a-464c-bb05-2f156da5652b","Type":"ContainerStarted","Data":"9da866380901805a991f6cd741a4ed8b94f2861b68385bc2161d2259e18b60e6"}
Nov 04 02:16:26.530331 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:26.530305    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt"
Nov 04 02:16:26.531728 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:26.531705    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk" event={"ID":"62b73809-c52f-4c4a-915d-f03e77df2068","Type":"ContainerStarted","Data":"5dd339c514d916178d34e74a1e92edf5ca7dec1d38b16275bbb7199b196a27eb"}
Nov 04 02:16:26.531815 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:26.531742    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk" event={"ID":"62b73809-c52f-4c4a-915d-f03e77df2068","Type":"ContainerStarted","Data":"82503543b0a22aa3fea3cb563c50afda8932ac65df6ec8ed99bf04863f4fad3d"}
Nov 04 02:16:26.531815 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:26.531761    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk"
Nov 04 02:16:26.647319 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:26.647270    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv" podStartSLOduration=2.832528383 podStartE2EDuration="3.647255528s" totalImagesPullingTime="814.727145ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:26.645001623 +0000 UTC m=+622.571224268" watchObservedRunningTime="2026-11-04 02:16:26.647255528 +0000 UTC m=+622.573478160"
Nov 04 02:16:26.760042 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:26.759595    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt" podStartSLOduration=1.759577052 podStartE2EDuration="1.759577052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:26.759496496 +0000 UTC m=+622.685719131" watchObservedRunningTime="2026-11-04 02:16:26.759577052 +0000 UTC m=+622.685799675"
Nov 04 02:16:26.875932 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:26.875836    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk" podStartSLOduration=2.8758224820000002 podStartE2EDuration="2.875822482s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:26.873354412 +0000 UTC m=+622.799577046" watchObservedRunningTime="2026-11-04 02:16:26.875822482 +0000 UTC m=+622.802045114"
Nov 04 02:16:27.174658 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:27.174577    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:27.358430 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:27.358396    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:27.498190 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:27.498119    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn"
Nov 04 02:16:27.552547 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:27.552518    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99"]
Nov 04 02:16:27.556480 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:27.556463    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99"
Nov 04 02:16:27.613926 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:27.613901    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99"]
Nov 04 02:16:27.615518 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:27.615498    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w6-9adedd\""
Nov 04 02:16:27.615757 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:27.615733    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w6-9adedd-dockercfg-nz8jd\""
Nov 04 02:16:27.657918 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:27.657882    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lkr7\" (UniqueName: \"kubernetes.io/projected/afaa9b17-4098-417a-aa81-eceb2ca22183-kube-api-access-9lkr7\") pod \"payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99\" (UID: \"afaa9b17-4098-417a-aa81-eceb2ca22183\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99"
Nov 04 02:16:27.658642 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:27.658126    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afaa9b17-4098-417a-aa81-eceb2ca22183-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99\" (UID: \"afaa9b17-4098-417a-aa81-eceb2ca22183\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99"
Nov 04 02:16:27.666499 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:27.666468    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2"]
Nov 04 02:16:27.679115 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:27.671500    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2"
Nov 04 02:16:27.729370 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:27.729342    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2"]
Nov 04 02:16:27.758982 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:27.758949    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lkr7\" (UniqueName: \"kubernetes.io/projected/afaa9b17-4098-417a-aa81-eceb2ca22183-kube-api-access-9lkr7\") pod \"payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99\" (UID: \"afaa9b17-4098-417a-aa81-eceb2ca22183\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99"
Nov 04 02:16:27.759162 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:27.759045    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr26w\" (UniqueName: \"kubernetes.io/projected/75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a-kube-api-access-pr26w\") pod \"payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2\" (UID: \"75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2"
Nov 04 02:16:27.759162 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:27.759087    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afaa9b17-4098-417a-aa81-eceb2ca22183-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99\" (UID: \"afaa9b17-4098-417a-aa81-eceb2ca22183\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99"
Nov 04 02:16:27.759297 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:27.759266    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2\" (UID: \"75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2"
Nov 04 02:16:27.759874 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:27.759852    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afaa9b17-4098-417a-aa81-eceb2ca22183-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99\" (UID: \"afaa9b17-4098-417a-aa81-eceb2ca22183\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99"
Nov 04 02:16:27.820088 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:27.820063    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lkr7\" (UniqueName: \"kubernetes.io/projected/afaa9b17-4098-417a-aa81-eceb2ca22183-kube-api-access-9lkr7\") pod \"payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99\" (UID: \"afaa9b17-4098-417a-aa81-eceb2ca22183\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99"
Nov 04 02:16:27.860275 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:27.860250    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr26w\" (UniqueName: \"kubernetes.io/projected/75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a-kube-api-access-pr26w\") pod \"payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2\" (UID: \"75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2"
Nov 04 02:16:27.860389 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:27.860305    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2\" (UID: \"75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2"
Nov 04 02:16:27.860911 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:27.860893    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2\" (UID: \"75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2"
Nov 04 02:16:27.869132 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:27.869089    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99"
Nov 04 02:16:27.923417 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:27.923382    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr26w\" (UniqueName: \"kubernetes.io/projected/75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a-kube-api-access-pr26w\") pod \"payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2\" (UID: \"75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2"
Nov 04 02:16:27.988614 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:27.988568    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2"
Nov 04 02:16:28.241794 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:28.241767    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99"]
Nov 04 02:16:28.243590 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:16:28.243563    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafaa9b17_4098_417a_aa81_eceb2ca22183.slice/crio-a479af58cf99b26a2b90762db9adf7c8c244fc1ce4b0ef98082b1d66a9b90804 WatchSource:0}: Error finding container a479af58cf99b26a2b90762db9adf7c8c244fc1ce4b0ef98082b1d66a9b90804: Status 404 returned error can't find the container with id a479af58cf99b26a2b90762db9adf7c8c244fc1ce4b0ef98082b1d66a9b90804
Nov 04 02:16:28.366794 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:28.366759    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2"]
Nov 04 02:16:28.369358 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:16:28.369331    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75a1aa0a_2af0_4dc7_8a0c_8e0ef2a9871a.slice/crio-3a775e02e4c9329370a5c25ed023124cdd6a167bf5ff9064b4627cd1e590ae89 WatchSource:0}: Error finding container 3a775e02e4c9329370a5c25ed023124cdd6a167bf5ff9064b4627cd1e590ae89: Status 404 returned error can't find the container with id 3a775e02e4c9329370a5c25ed023124cdd6a167bf5ff9064b4627cd1e590ae89
Nov 04 02:16:28.543935 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:28.543895    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99" event={"ID":"afaa9b17-4098-417a-aa81-eceb2ca22183","Type":"ContainerStarted","Data":"49192ea6e0209f3953cc846c74febfdca10a77b87f88571430946db5c31795b0"}
Nov 04 02:16:28.544139 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:28.543965    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99" event={"ID":"afaa9b17-4098-417a-aa81-eceb2ca22183","Type":"ContainerStarted","Data":"a479af58cf99b26a2b90762db9adf7c8c244fc1ce4b0ef98082b1d66a9b90804"}
Nov 04 02:16:28.544139 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:28.544075    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99"
Nov 04 02:16:28.545536 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:28.545513    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2" event={"ID":"75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a","Type":"ContainerStarted","Data":"c5130a3402ecc15c5869b76950d0649e932355bd4ac2e4fecfda251837d9b5c8"}
Nov 04 02:16:28.545619 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:28.545554    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2" event={"ID":"75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a","Type":"ContainerStarted","Data":"3a775e02e4c9329370a5c25ed023124cdd6a167bf5ff9064b4627cd1e590ae89"}
Nov 04 02:16:28.545619 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:28.545578    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2"
Nov 04 02:16:28.661156 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:28.661037    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99" podStartSLOduration=1.6610230270000002 podStartE2EDuration="1.661023027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:28.658718871 +0000 UTC m=+624.584941505" watchObservedRunningTime="2026-11-04 02:16:28.661023027 +0000 UTC m=+624.587245660"
Nov 04 02:16:28.775743 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:28.775690    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2" podStartSLOduration=1.7756765749999999 podStartE2EDuration="1.775676575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:28.773282868 +0000 UTC m=+624.699505502" watchObservedRunningTime="2026-11-04 02:16:28.775676575 +0000 UTC m=+624.701899258"
Nov 04 02:16:28.842249 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:28.842212    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs"]
Nov 04 02:16:28.846188 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:28.846166    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs"
Nov 04 02:16:28.898427 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:28.898396    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-9adedd-serving-cert\""
Nov 04 02:16:28.906243 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:28.906210    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs"]
Nov 04 02:16:28.977723 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:28.977623    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs\" (UID: \"269fe5b3-6355-4cdc-a33a-fc4bfe897a79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs"
Nov 04 02:16:28.977899 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:28.977754    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-maas-api-tls\") pod \"maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs\" (UID: \"269fe5b3-6355-4cdc-a33a-fc4bfe897a79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs"
Nov 04 02:16:28.977899 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:28.977817    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9qc4\" (UniqueName: \"kubernetes.io/projected/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-kube-api-access-v9qc4\") pod \"maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs\" (UID: \"269fe5b3-6355-4cdc-a33a-fc4bfe897a79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs"
Nov 04 02:16:28.978083 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:28.977953    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-metrics-certs\") pod \"maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs\" (UID: \"269fe5b3-6355-4cdc-a33a-fc4bfe897a79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs"
Nov 04 02:16:29.075953 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:29.075891    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:29.079132 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:29.079078    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9qc4\" (UniqueName: \"kubernetes.io/projected/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-kube-api-access-v9qc4\") pod \"maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs\" (UID: \"269fe5b3-6355-4cdc-a33a-fc4bfe897a79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs"
Nov 04 02:16:29.079251 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:29.079146    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-metrics-certs\") pod \"maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs\" (UID: \"269fe5b3-6355-4cdc-a33a-fc4bfe897a79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs"
Nov 04 02:16:29.079251 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:29.079204    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs\" (UID: \"269fe5b3-6355-4cdc-a33a-fc4bfe897a79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs"
Nov 04 02:16:29.079356 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:29.079256    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-maas-api-tls\") pod \"maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs\" (UID: \"269fe5b3-6355-4cdc-a33a-fc4bfe897a79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs"
Nov 04 02:16:29.079407 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:16:29.079355    4969 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-serving-cert: secret "maas-api-e2e-worker-w6-9adedd-serving-cert" not found
Nov 04 02:16:29.079459 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:16:29.079408    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-maas-api-tls podName:269fe5b3-6355-4cdc-a33a-fc4bfe897a79 nodeName:}" failed. No retries permitted until 2026-11-04 02:16:29.57939217 +0000 UTC m=+625.505614796 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-maas-api-tls") pod "maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs" (UID: "269fe5b3-6355-4cdc-a33a-fc4bfe897a79") : secret "maas-api-e2e-worker-w6-9adedd-serving-cert" not found
Nov 04 02:16:29.079928 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:29.079904    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs\" (UID: \"269fe5b3-6355-4cdc-a33a-fc4bfe897a79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs"
Nov 04 02:16:29.081473 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:29.081448    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-metrics-certs\") pod \"maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs\" (UID: \"269fe5b3-6355-4cdc-a33a-fc4bfe897a79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs"
Nov 04 02:16:29.142298 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:29.142271    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9qc4\" (UniqueName: \"kubernetes.io/projected/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-kube-api-access-v9qc4\") pod \"maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs\" (UID: \"269fe5b3-6355-4cdc-a33a-fc4bfe897a79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs"
Nov 04 02:16:29.259278 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:29.259250    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:29.437083 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:29.437053    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:29.474116 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:29.474071    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc"
Nov 04 02:16:29.474261 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:29.474250    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl"
Nov 04 02:16:29.584943 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:29.584865    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-maas-api-tls\") pod \"maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs\" (UID: \"269fe5b3-6355-4cdc-a33a-fc4bfe897a79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs"
Nov 04 02:16:29.587486 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:29.587460    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-maas-api-tls\") pod \"maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs\" (UID: \"269fe5b3-6355-4cdc-a33a-fc4bfe897a79\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs"
Nov 04 02:16:29.622665 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:29.622632    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:29.761276 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:29.761244    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs"
Nov 04 02:16:29.802576 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:29.802546    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:29.989419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:29.989326    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:30.060871 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:30.060840    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj"]
Nov 04 02:16:30.066026 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:30.066005    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj"
Nov 04 02:16:30.119410 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:30.119332    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-9b0f0d-dockercfg-h5m8c\""
Nov 04 02:16:30.119410 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:30.119368    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-9b0f0d\""
Nov 04 02:16:30.122929 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:30.122908    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj"]
Nov 04 02:16:30.186500 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:30.186473    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs"]
Nov 04 02:16:30.187663 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:16:30.187634    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod269fe5b3_6355_4cdc_a33a_fc4bfe897a79.slice/crio-7e8b1296090622f5b4cd2a6562ebedf6cffac53342d623c725745b53ca55119f WatchSource:0}: Error finding container 7e8b1296090622f5b4cd2a6562ebedf6cffac53342d623c725745b53ca55119f: Status 404 returned error can't find the container with id 7e8b1296090622f5b4cd2a6562ebedf6cffac53342d623c725745b53ca55119f
Nov 04 02:16:30.191771 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:30.191745    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqq24\" (UniqueName: \"kubernetes.io/projected/43d4ca4c-da17-4f49-898e-d08e931bc93b-kube-api-access-jqq24\") pod \"payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj\" (UID: \"43d4ca4c-da17-4f49-898e-d08e931bc93b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj"
Nov 04 02:16:30.191950 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:30.191928    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43d4ca4c-da17-4f49-898e-d08e931bc93b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj\" (UID: \"43d4ca4c-da17-4f49-898e-d08e931bc93b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj"
Nov 04 02:16:30.204409 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:30.204384    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd"]
Nov 04 02:16:30.213509 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:30.213491    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd"
Nov 04 02:16:30.266622 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:30.266592    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd"]
Nov 04 02:16:30.292750 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:30.292723    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqq24\" (UniqueName: \"kubernetes.io/projected/43d4ca4c-da17-4f49-898e-d08e931bc93b-kube-api-access-jqq24\") pod \"payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj\" (UID: \"43d4ca4c-da17-4f49-898e-d08e931bc93b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj"
Nov 04 02:16:30.292871 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:30.292779    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/440b09dc-d2a6-4879-8b71-e189dd565c27-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd\" (UID: \"440b09dc-d2a6-4879-8b71-e189dd565c27\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd"
Nov 04 02:16:30.292924 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:30.292882    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6k7h\" (UniqueName: \"kubernetes.io/projected/440b09dc-d2a6-4879-8b71-e189dd565c27-kube-api-access-h6k7h\") pod \"payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd\" (UID: \"440b09dc-d2a6-4879-8b71-e189dd565c27\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd"
Nov 04 02:16:30.292962 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:30.292940    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43d4ca4c-da17-4f49-898e-d08e931bc93b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj\" (UID: \"43d4ca4c-da17-4f49-898e-d08e931bc93b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj"
Nov 04 02:16:30.293613 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:30.293596    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43d4ca4c-da17-4f49-898e-d08e931bc93b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj\" (UID: \"43d4ca4c-da17-4f49-898e-d08e931bc93b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj"
Nov 04 02:16:30.354060 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:30.354029    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqq24\" (UniqueName: \"kubernetes.io/projected/43d4ca4c-da17-4f49-898e-d08e931bc93b-kube-api-access-jqq24\") pod \"payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj\" (UID: \"43d4ca4c-da17-4f49-898e-d08e931bc93b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj"
Nov 04 02:16:30.380886 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:30.380854    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj"
Nov 04 02:16:30.394404 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:30.394378    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/440b09dc-d2a6-4879-8b71-e189dd565c27-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd\" (UID: \"440b09dc-d2a6-4879-8b71-e189dd565c27\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd"
Nov 04 02:16:30.394548 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:30.394533    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6k7h\" (UniqueName: \"kubernetes.io/projected/440b09dc-d2a6-4879-8b71-e189dd565c27-kube-api-access-h6k7h\") pod \"payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd\" (UID: \"440b09dc-d2a6-4879-8b71-e189dd565c27\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd"
Nov 04 02:16:30.395297 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:30.395277    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/440b09dc-d2a6-4879-8b71-e189dd565c27-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd\" (UID: \"440b09dc-d2a6-4879-8b71-e189dd565c27\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd"
Nov 04 02:16:30.455338 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:30.455312    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6k7h\" (UniqueName: \"kubernetes.io/projected/440b09dc-d2a6-4879-8b71-e189dd565c27-kube-api-access-h6k7h\") pod \"payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd\" (UID: \"440b09dc-d2a6-4879-8b71-e189dd565c27\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd"
Nov 04 02:16:30.527283 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:30.527226    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd"
Nov 04 02:16:30.558862 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:30.558829    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs" event={"ID":"269fe5b3-6355-4cdc-a33a-fc4bfe897a79","Type":"ContainerStarted","Data":"7e8b1296090622f5b4cd2a6562ebedf6cffac53342d623c725745b53ca55119f"}
Nov 04 02:16:30.754925 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:30.754903    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj"]
Nov 04 02:16:30.756587 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:16:30.756559    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43d4ca4c_da17_4f49_898e_d08e931bc93b.slice/crio-862235b41507605a3339185d3801c94a202fb0b03b3d46f6d46aef688be5d011 WatchSource:0}: Error finding container 862235b41507605a3339185d3801c94a202fb0b03b3d46f6d46aef688be5d011: Status 404 returned error can't find the container with id 862235b41507605a3339185d3801c94a202fb0b03b3d46f6d46aef688be5d011
Nov 04 02:16:30.900410 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:30.900388    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd"]
Nov 04 02:16:30.901936 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:16:30.901907    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod440b09dc_d2a6_4879_8b71_e189dd565c27.slice/crio-02afb26fe63053fa97efe8165eaf09c69f075f971de6820c685e6f6bc1c349ed WatchSource:0}: Error finding container 02afb26fe63053fa97efe8165eaf09c69f075f971de6820c685e6f6bc1c349ed: Status 404 returned error can't find the container with id 02afb26fe63053fa97efe8165eaf09c69f075f971de6820c685e6f6bc1c349ed
Nov 04 02:16:31.565801 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:31.565769    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs" event={"ID":"269fe5b3-6355-4cdc-a33a-fc4bfe897a79","Type":"ContainerStarted","Data":"7275ca47f22b120ffcbe01eb6269bb0bc8579392a40fba134996cb809346946e"}
Nov 04 02:16:31.565971 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:31.565952    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs"
Nov 04 02:16:31.567478 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:31.567453    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj" event={"ID":"43d4ca4c-da17-4f49-898e-d08e931bc93b","Type":"ContainerStarted","Data":"b107c0546c13db77241de79405a565ae385d1c4abe27b87bda64e7ebca588907"}
Nov 04 02:16:31.567576 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:31.567503    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj" event={"ID":"43d4ca4c-da17-4f49-898e-d08e931bc93b","Type":"ContainerStarted","Data":"862235b41507605a3339185d3801c94a202fb0b03b3d46f6d46aef688be5d011"}
Nov 04 02:16:31.567576 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:31.567538    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj"
Nov 04 02:16:31.569059 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:31.569036    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd" event={"ID":"440b09dc-d2a6-4879-8b71-e189dd565c27","Type":"ContainerStarted","Data":"8371a6c2aace7aff4430f2a356f6ddfc0591242c349230f0e343cbfa95560d44"}
Nov 04 02:16:31.569150 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:31.569075    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd" event={"ID":"440b09dc-d2a6-4879-8b71-e189dd565c27","Type":"ContainerStarted","Data":"02afb26fe63053fa97efe8165eaf09c69f075f971de6820c685e6f6bc1c349ed"}
Nov 04 02:16:31.569150 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:31.569111    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd"
Nov 04 02:16:31.685010 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:31.684951    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs" podStartSLOduration=2.814476627 podStartE2EDuration="3.684937443s" totalImagesPullingTime="870.460816ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:31.682699522 +0000 UTC m=+627.608922157" watchObservedRunningTime="2026-11-04 02:16:31.684937443 +0000 UTC m=+627.611160076"
Nov 04 02:16:31.799686 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:31.799616    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd" podStartSLOduration=1.799601711 podStartE2EDuration="1.799601711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:31.797042929 +0000 UTC m=+627.723265555" watchObservedRunningTime="2026-11-04 02:16:31.799601711 +0000 UTC m=+627.725824344"
Nov 04 02:16:31.912837 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:31.912718    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj" podStartSLOduration=1.912702404 podStartE2EDuration="1.912702404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:31.911650574 +0000 UTC m=+627.837873206" watchObservedRunningTime="2026-11-04 02:16:31.912702404 +0000 UTC m=+627.838925038"
Nov 04 02:16:32.542564 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:32.542529    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv"
Nov 04 02:16:33.916945 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:33.916916    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh"]
Nov 04 02:16:33.921029 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:33.921012    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh"
Nov 04 02:16:33.975948 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:33.975923    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-9b0f0d-serving-cert\""
Nov 04 02:16:34.027815 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:34.027780    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0858fde0-e137-4c9a-a19b-99f70732dcf5-metrics-certs\") pod \"maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh\" (UID: \"0858fde0-e137-4c9a-a19b-99f70732dcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh"
Nov 04 02:16:34.027977 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:34.027936    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpjzm\" (UniqueName: \"kubernetes.io/projected/0858fde0-e137-4c9a-a19b-99f70732dcf5-kube-api-access-qpjzm\") pod \"maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh\" (UID: \"0858fde0-e137-4c9a-a19b-99f70732dcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh"
Nov 04 02:16:34.028043 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:34.027989    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0858fde0-e137-4c9a-a19b-99f70732dcf5-maas-api-tls\") pod \"maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh\" (UID: \"0858fde0-e137-4c9a-a19b-99f70732dcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh"
Nov 04 02:16:34.028123 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:34.028047    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0858fde0-e137-4c9a-a19b-99f70732dcf5-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh\" (UID: \"0858fde0-e137-4c9a-a19b-99f70732dcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh"
Nov 04 02:16:34.030168 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:34.030110    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh"]
Nov 04 02:16:34.082527 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:34.082485    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:34.128807 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:34.128769    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0858fde0-e137-4c9a-a19b-99f70732dcf5-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh\" (UID: \"0858fde0-e137-4c9a-a19b-99f70732dcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh"
Nov 04 02:16:34.128997 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:34.128822    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0858fde0-e137-4c9a-a19b-99f70732dcf5-metrics-certs\") pod \"maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh\" (UID: \"0858fde0-e137-4c9a-a19b-99f70732dcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh"
Nov 04 02:16:34.128997 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:34.128907    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpjzm\" (UniqueName: \"kubernetes.io/projected/0858fde0-e137-4c9a-a19b-99f70732dcf5-kube-api-access-qpjzm\") pod \"maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh\" (UID: \"0858fde0-e137-4c9a-a19b-99f70732dcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh"
Nov 04 02:16:34.128997 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:34.128944    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0858fde0-e137-4c9a-a19b-99f70732dcf5-maas-api-tls\") pod \"maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh\" (UID: \"0858fde0-e137-4c9a-a19b-99f70732dcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh"
Nov 04 02:16:34.129193 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:16:34.129042    4969 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-serving-cert: secret "maas-api-e2e-worker-w2-9b0f0d-serving-cert" not found
Nov 04 02:16:34.129193 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:16:34.129118    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0858fde0-e137-4c9a-a19b-99f70732dcf5-maas-api-tls podName:0858fde0-e137-4c9a-a19b-99f70732dcf5 nodeName:}" failed. No retries permitted until 2026-11-04 02:16:34.62908293 +0000 UTC m=+630.555305555 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0858fde0-e137-4c9a-a19b-99f70732dcf5-maas-api-tls") pod "maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh" (UID: "0858fde0-e137-4c9a-a19b-99f70732dcf5") : secret "maas-api-e2e-worker-w2-9b0f0d-serving-cert" not found
Nov 04 02:16:34.129563 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:34.129539    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0858fde0-e137-4c9a-a19b-99f70732dcf5-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh\" (UID: \"0858fde0-e137-4c9a-a19b-99f70732dcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh"
Nov 04 02:16:34.131387 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:34.131364    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0858fde0-e137-4c9a-a19b-99f70732dcf5-metrics-certs\") pod \"maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh\" (UID: \"0858fde0-e137-4c9a-a19b-99f70732dcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh"
Nov 04 02:16:34.190337 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:34.190278    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpjzm\" (UniqueName: \"kubernetes.io/projected/0858fde0-e137-4c9a-a19b-99f70732dcf5-kube-api-access-qpjzm\") pod \"maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh\" (UID: \"0858fde0-e137-4c9a-a19b-99f70732dcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh"
Nov 04 02:16:34.259008 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:34.258975    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:34.263360 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:16:34.263326    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:16:34.434105 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:34.434057    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:34.515655 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:34.515628    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx"
Nov 04 02:16:34.515844 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:34.515705    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt"
Nov 04 02:16:34.610577 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:34.610544    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:34.633842 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:34.633811    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0858fde0-e137-4c9a-a19b-99f70732dcf5-maas-api-tls\") pod \"maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh\" (UID: \"0858fde0-e137-4c9a-a19b-99f70732dcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh"
Nov 04 02:16:34.636538 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:34.636515    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0858fde0-e137-4c9a-a19b-99f70732dcf5-maas-api-tls\") pod \"maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh\" (UID: \"0858fde0-e137-4c9a-a19b-99f70732dcf5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh"
Nov 04 02:16:34.794802 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:34.794727    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:34.834791 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:34.834760    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh"
Nov 04 02:16:35.145208 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:35.145133    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc"]
Nov 04 02:16:35.150270 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:35.150251    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc"
Nov 04 02:16:35.204080 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:35.204022    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-8cf267-dockercfg-zz682\""
Nov 04 02:16:35.207753 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:35.207728    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc"]
Nov 04 02:16:35.239659 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:35.239627    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw2zv\" (UniqueName: \"kubernetes.io/projected/756a3073-fe07-4efa-9469-3862ab9350d4-kube-api-access-qw2zv\") pod \"payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc\" (UID: \"756a3073-fe07-4efa-9469-3862ab9350d4\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc"
Nov 04 02:16:35.239777 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:35.239676    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/756a3073-fe07-4efa-9469-3862ab9350d4-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc\" (UID: \"756a3073-fe07-4efa-9469-3862ab9350d4\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc"
Nov 04 02:16:35.268530 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:35.268504    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-8cf267\""
Nov 04 02:16:35.272149 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:35.272125    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh"]
Nov 04 02:16:35.273290 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:16:35.273269    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0858fde0_e137_4c9a_a19b_99f70732dcf5.slice/crio-ddf24e3af5835934a6e529563a4554fef457a406c3e86c654b1bd73c19cb9ca8 WatchSource:0}: Error finding container ddf24e3af5835934a6e529563a4554fef457a406c3e86c654b1bd73c19cb9ca8: Status 404 returned error can't find the container with id ddf24e3af5835934a6e529563a4554fef457a406c3e86c654b1bd73c19cb9ca8
Nov 04 02:16:35.340370 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:35.340341    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw2zv\" (UniqueName: \"kubernetes.io/projected/756a3073-fe07-4efa-9469-3862ab9350d4-kube-api-access-qw2zv\") pod \"payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc\" (UID: \"756a3073-fe07-4efa-9469-3862ab9350d4\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc"
Nov 04 02:16:35.340467 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:35.340386    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/756a3073-fe07-4efa-9469-3862ab9350d4-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc\" (UID: \"756a3073-fe07-4efa-9469-3862ab9350d4\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc"
Nov 04 02:16:35.341089 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:35.341071    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/756a3073-fe07-4efa-9469-3862ab9350d4-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc\" (UID: \"756a3073-fe07-4efa-9469-3862ab9350d4\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc"
Nov 04 02:16:35.401373 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:35.401304    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw2zv\" (UniqueName: \"kubernetes.io/projected/756a3073-fe07-4efa-9469-3862ab9350d4-kube-api-access-qw2zv\") pod \"payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc\" (UID: \"756a3073-fe07-4efa-9469-3862ab9350d4\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc"
Nov 04 02:16:35.464056 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:35.464028    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc"
Nov 04 02:16:35.594653 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:35.594612    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh" event={"ID":"0858fde0-e137-4c9a-a19b-99f70732dcf5","Type":"ContainerStarted","Data":"ddf24e3af5835934a6e529563a4554fef457a406c3e86c654b1bd73c19cb9ca8"}
Nov 04 02:16:35.838605 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:35.838576    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc"]
Nov 04 02:16:35.839737 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:16:35.839713    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod756a3073_fe07_4efa_9469_3862ab9350d4.slice/crio-58d5a3a98b96e7531a19e4478eb4169da5ac941a9f7065e76b4e6b58d06b9d92 WatchSource:0}: Error finding container 58d5a3a98b96e7531a19e4478eb4169da5ac941a9f7065e76b4e6b58d06b9d92: Status 404 returned error can't find the container with id 58d5a3a98b96e7531a19e4478eb4169da5ac941a9f7065e76b4e6b58d06b9d92
Nov 04 02:16:36.604122 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:36.604070    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh" event={"ID":"0858fde0-e137-4c9a-a19b-99f70732dcf5","Type":"ContainerStarted","Data":"584660ccaf82e36a5a324c4b8b905a22a6f9925c59f4301372245452515ca788"}
Nov 04 02:16:36.604601 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:36.604574    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh"
Nov 04 02:16:36.605924 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:36.605903    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc" event={"ID":"756a3073-fe07-4efa-9469-3862ab9350d4","Type":"ContainerStarted","Data":"557bc90183d721cc7f94097a91683ad14397aede4c52cd541be0b45f2307ae4a"}
Nov 04 02:16:36.606024 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:36.605947    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc" event={"ID":"756a3073-fe07-4efa-9469-3862ab9350d4","Type":"ContainerStarted","Data":"58d5a3a98b96e7531a19e4478eb4169da5ac941a9f7065e76b4e6b58d06b9d92"}
Nov 04 02:16:36.606024 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:36.605977    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc"
Nov 04 02:16:36.727686 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:36.727632    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh" podStartSLOduration=2.846162696 podStartE2EDuration="3.727616407s" totalImagesPullingTime="881.453711ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:36.725945098 +0000 UTC m=+632.652167742" watchObservedRunningTime="2026-11-04 02:16:36.727616407 +0000 UTC m=+632.653839039"
Nov 04 02:16:36.851111 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:36.843229    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc" podStartSLOduration=1.8432088420000001 podStartE2EDuration="1.843208842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:36.841561726 +0000 UTC m=+632.767784359" watchObservedRunningTime="2026-11-04 02:16:36.843208842 +0000 UTC m=+632.769431478"
Nov 04 02:16:37.540260 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:37.540232    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk"
Nov 04 02:16:37.540456 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:37.540312    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt"
Nov 04 02:16:37.579278 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:37.579255    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs"
Nov 04 02:16:39.050723 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:39.050690    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:39.236693 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:39.236659    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:39.425160 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:39.425070    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:39.554133 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:39.554104    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99"
Nov 04 02:16:39.554295 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:39.554283    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2"
Nov 04 02:16:39.614363 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:39.614334    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:39.813222 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:39.813130    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:42.577862 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:42.577832    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj"
Nov 04 02:16:42.578527 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:42.578177    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd"
Nov 04 02:16:42.616745 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:42.616722    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh"
Nov 04 02:16:44.116190 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:44.116135    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:47.614900 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:47.614865    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc"
Nov 04 02:16:48.743596 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:48.743555    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2"]
Nov 04 02:16:48.747715 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:48.747695    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2"
Nov 04 02:16:48.791548 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:48.791515    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50d5e35c-7ec2-4285-b10a-82e94e9f608f-maas-api-tls\") pod \"maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2\" (UID: \"50d5e35c-7ec2-4285-b10a-82e94e9f608f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2"
Nov 04 02:16:48.791692 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:48.791594    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcjc6\" (UniqueName: \"kubernetes.io/projected/50d5e35c-7ec2-4285-b10a-82e94e9f608f-kube-api-access-fcjc6\") pod \"maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2\" (UID: \"50d5e35c-7ec2-4285-b10a-82e94e9f608f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2"
Nov 04 02:16:48.791763 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:48.791705    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50d5e35c-7ec2-4285-b10a-82e94e9f608f-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2\" (UID: \"50d5e35c-7ec2-4285-b10a-82e94e9f608f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2"
Nov 04 02:16:48.791763 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:48.791752    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50d5e35c-7ec2-4285-b10a-82e94e9f608f-metrics-certs\") pod \"maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2\" (UID: \"50d5e35c-7ec2-4285-b10a-82e94e9f608f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2"
Nov 04 02:16:48.849354 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:48.849325    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-5a27ba-serving-cert\""
Nov 04 02:16:48.852113 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:48.852064    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2"]
Nov 04 02:16:48.892820 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:48.892786    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50d5e35c-7ec2-4285-b10a-82e94e9f608f-maas-api-tls\") pod \"maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2\" (UID: \"50d5e35c-7ec2-4285-b10a-82e94e9f608f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2"
Nov 04 02:16:48.893211 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:48.893186    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcjc6\" (UniqueName: \"kubernetes.io/projected/50d5e35c-7ec2-4285-b10a-82e94e9f608f-kube-api-access-fcjc6\") pod \"maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2\" (UID: \"50d5e35c-7ec2-4285-b10a-82e94e9f608f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2"
Nov 04 02:16:48.893332 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:16:48.893076    4969 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-serving-cert: secret "maas-api-e2e-worker-w3-5a27ba-serving-cert" not found
Nov 04 02:16:48.893395 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:16:48.893348    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50d5e35c-7ec2-4285-b10a-82e94e9f608f-maas-api-tls podName:50d5e35c-7ec2-4285-b10a-82e94e9f608f nodeName:}" failed. No retries permitted until 2026-11-04 02:16:49.393325726 +0000 UTC m=+645.319548354 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/50d5e35c-7ec2-4285-b10a-82e94e9f608f-maas-api-tls") pod "maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2" (UID: "50d5e35c-7ec2-4285-b10a-82e94e9f608f") : secret "maas-api-e2e-worker-w3-5a27ba-serving-cert" not found
Nov 04 02:16:48.893697 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:48.893679    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50d5e35c-7ec2-4285-b10a-82e94e9f608f-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2\" (UID: \"50d5e35c-7ec2-4285-b10a-82e94e9f608f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2"
Nov 04 02:16:48.893878 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:48.893863    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50d5e35c-7ec2-4285-b10a-82e94e9f608f-metrics-certs\") pod \"maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2\" (UID: \"50d5e35c-7ec2-4285-b10a-82e94e9f608f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2"
Nov 04 02:16:48.894597 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:48.894574    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50d5e35c-7ec2-4285-b10a-82e94e9f608f-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2\" (UID: \"50d5e35c-7ec2-4285-b10a-82e94e9f608f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2"
Nov 04 02:16:48.897496 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:48.897477    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50d5e35c-7ec2-4285-b10a-82e94e9f608f-metrics-certs\") pod \"maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2\" (UID: \"50d5e35c-7ec2-4285-b10a-82e94e9f608f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2"
Nov 04 02:16:48.954328 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:48.954301    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcjc6\" (UniqueName: \"kubernetes.io/projected/50d5e35c-7ec2-4285-b10a-82e94e9f608f-kube-api-access-fcjc6\") pod \"maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2\" (UID: \"50d5e35c-7ec2-4285-b10a-82e94e9f608f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2"
Nov 04 02:16:48.975979 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:48.975955    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:49.152650 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:49.152596    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:49.398120 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:49.398075    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50d5e35c-7ec2-4285-b10a-82e94e9f608f-maas-api-tls\") pod \"maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2\" (UID: \"50d5e35c-7ec2-4285-b10a-82e94e9f608f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2"
Nov 04 02:16:49.400451 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:49.400428    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50d5e35c-7ec2-4285-b10a-82e94e9f608f-maas-api-tls\") pod \"maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2\" (UID: \"50d5e35c-7ec2-4285-b10a-82e94e9f608f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2"
Nov 04 02:16:49.661580 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:49.661543    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2"
Nov 04 02:16:50.039535 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:50.039510    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2"]
Nov 04 02:16:50.042640 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:16:50.042604    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50d5e35c_7ec2_4285_b10a_82e94e9f608f.slice/crio-2111d22f4bf8321c18a96f7f80323e6265e83e5f37dafe9eed7ea1da934a9cfa WatchSource:0}: Error finding container 2111d22f4bf8321c18a96f7f80323e6265e83e5f37dafe9eed7ea1da934a9cfa: Status 404 returned error can't find the container with id 2111d22f4bf8321c18a96f7f80323e6265e83e5f37dafe9eed7ea1da934a9cfa
Nov 04 02:16:50.044166 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:50.044150    4969 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 02:16:50.701493 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:50.701451    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2" event={"ID":"50d5e35c-7ec2-4285-b10a-82e94e9f608f","Type":"ContainerStarted","Data":"2111d22f4bf8321c18a96f7f80323e6265e83e5f37dafe9eed7ea1da934a9cfa"}
Nov 04 02:16:51.709357 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:51.709321    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2" event={"ID":"50d5e35c-7ec2-4285-b10a-82e94e9f608f","Type":"ContainerStarted","Data":"2a56e0e67034e605b5629a2b72b7ae58b46d808627bc90f474947d63a66fddf1"}
Nov 04 02:16:51.710217 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:51.710191    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2"
Nov 04 02:16:51.828503 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:51.828440    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2" podStartSLOduration=3.032174246 podStartE2EDuration="3.828420487s" totalImagesPullingTime="796.246241ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:51.826542594 +0000 UTC m=+647.752765231" watchObservedRunningTime="2026-11-04 02:16:51.828420487 +0000 UTC m=+647.754643122"
Nov 04 02:16:53.879710 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:53.879674    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:54.071969 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:54.071930    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:54.254815 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:54.254737    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:57.564838 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:57.564800    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:16:58.728416 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:58.728390    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2"
Nov 04 02:16:58.928055 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:16:58.928024    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:03.838736 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:03.838700    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:04.262904 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:17:04.262868    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:17:07.670512 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:07.670481    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:08.361902 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:08.361858    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:08.560678 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:08.560646    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:09.069184 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:09.069147    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:11.778274 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:11.778246    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:12.933779 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:12.933729    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:13.507221 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:13.507190    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:13.716948 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:13.716912    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:14.442879 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:14.442840    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:18.673385 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:18.673354    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:19.296629 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:19.296598    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:19.471321 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:19.471271    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:19.762959 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:19.762931    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:25.011231 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:25.011202    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:25.272292 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:25.272214    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:25.621675 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:25.621600    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:25.820506 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:25.820474    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:29.054508 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:29.054471    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:29.690971 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:29.690937    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:29.876220 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:29.876189    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:30.897812 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:30.897780    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:34.261225 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:17:34.261183    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:17:36.030032 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:36.029995    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:36.718667 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:36.718634    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:38.447678 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:38.447642    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:39.043483 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:39.043449    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:39.245619 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:39.245578    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:40.184452 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:40.184420    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:41.392060 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:41.392028    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:45.885209 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:45.885180    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:46.615528 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:46.615493    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:46.932226 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:46.932149    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:51.545162 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:51.545128    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:51.720436 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:51.720391    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:56.541552 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:56.541512    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:17:57.920807 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:17:57.920771    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:00.661030 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:00.660986    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"]
Nov 04 02:18:00.665283 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:00.665260    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"
Nov 04 02:18:00.731945 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:00.731906    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzb2w\" (UniqueName: \"kubernetes.io/projected/a3d654e3-9e45-46e3-bad8-cde2b543684f-kube-api-access-zzb2w\") pod \"simulator-w1-8cf267-kserve-f647797c8-qrfr9\" (UID: \"a3d654e3-9e45-46e3-bad8-cde2b543684f\") " pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"
Nov 04 02:18:00.732086 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:00.732051    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3d654e3-9e45-46e3-bad8-cde2b543684f-model-cache\") pod \"simulator-w1-8cf267-kserve-f647797c8-qrfr9\" (UID: \"a3d654e3-9e45-46e3-bad8-cde2b543684f\") " pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"
Nov 04 02:18:00.732219 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:00.732198    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3d654e3-9e45-46e3-bad8-cde2b543684f-tmp-dir\") pod \"simulator-w1-8cf267-kserve-f647797c8-qrfr9\" (UID: \"a3d654e3-9e45-46e3-bad8-cde2b543684f\") " pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"
Nov 04 02:18:00.732277 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:00.732236    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3d654e3-9e45-46e3-bad8-cde2b543684f-home\") pod \"simulator-w1-8cf267-kserve-f647797c8-qrfr9\" (UID: \"a3d654e3-9e45-46e3-bad8-cde2b543684f\") " pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"
Nov 04 02:18:00.732390 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:00.732368    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3d654e3-9e45-46e3-bad8-cde2b543684f-tls-certs\") pod \"simulator-w1-8cf267-kserve-f647797c8-qrfr9\" (UID: \"a3d654e3-9e45-46e3-bad8-cde2b543684f\") " pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"
Nov 04 02:18:00.732435 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:00.732408    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3d654e3-9e45-46e3-bad8-cde2b543684f-dshm\") pod \"simulator-w1-8cf267-kserve-f647797c8-qrfr9\" (UID: \"a3d654e3-9e45-46e3-bad8-cde2b543684f\") " pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"
Nov 04 02:18:00.770186 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:00.770161    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-8cf267\"/\"kube-root-ca.crt\""
Nov 04 02:18:00.770373 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:00.770352    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-8cf267\"/\"simulator-w1-8cf267-kserve-self-signed-certs\""
Nov 04 02:18:00.770451 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:00.770357    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-8cf267\"/\"default-dockercfg-7cb68\""
Nov 04 02:18:00.770451 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:00.770383    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-8cf267\"/\"openshift-service-ca.crt\""
Nov 04 02:18:00.772765 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:00.772744    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"]
Nov 04 02:18:00.803303 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:00.803280    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:00.833129 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:00.833073    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3d654e3-9e45-46e3-bad8-cde2b543684f-tls-certs\") pod \"simulator-w1-8cf267-kserve-f647797c8-qrfr9\" (UID: \"a3d654e3-9e45-46e3-bad8-cde2b543684f\") " pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"
Nov 04 02:18:00.833308 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:00.833141    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3d654e3-9e45-46e3-bad8-cde2b543684f-dshm\") pod \"simulator-w1-8cf267-kserve-f647797c8-qrfr9\" (UID: \"a3d654e3-9e45-46e3-bad8-cde2b543684f\") " pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"
Nov 04 02:18:00.833308 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:00.833209    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzb2w\" (UniqueName: \"kubernetes.io/projected/a3d654e3-9e45-46e3-bad8-cde2b543684f-kube-api-access-zzb2w\") pod \"simulator-w1-8cf267-kserve-f647797c8-qrfr9\" (UID: \"a3d654e3-9e45-46e3-bad8-cde2b543684f\") " pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"
Nov 04 02:18:00.833308 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:00.833278    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3d654e3-9e45-46e3-bad8-cde2b543684f-model-cache\") pod \"simulator-w1-8cf267-kserve-f647797c8-qrfr9\" (UID: \"a3d654e3-9e45-46e3-bad8-cde2b543684f\") " pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"
Nov 04 02:18:00.833486 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:00.833381    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3d654e3-9e45-46e3-bad8-cde2b543684f-tmp-dir\") pod \"simulator-w1-8cf267-kserve-f647797c8-qrfr9\" (UID: \"a3d654e3-9e45-46e3-bad8-cde2b543684f\") " pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"
Nov 04 02:18:00.833486 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:00.833411    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3d654e3-9e45-46e3-bad8-cde2b543684f-home\") pod \"simulator-w1-8cf267-kserve-f647797c8-qrfr9\" (UID: \"a3d654e3-9e45-46e3-bad8-cde2b543684f\") " pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"
Nov 04 02:18:00.833766 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:00.833745    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3d654e3-9e45-46e3-bad8-cde2b543684f-model-cache\") pod \"simulator-w1-8cf267-kserve-f647797c8-qrfr9\" (UID: \"a3d654e3-9e45-46e3-bad8-cde2b543684f\") " pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"
Nov 04 02:18:00.833872 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:00.833789    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3d654e3-9e45-46e3-bad8-cde2b543684f-tmp-dir\") pod \"simulator-w1-8cf267-kserve-f647797c8-qrfr9\" (UID: \"a3d654e3-9e45-46e3-bad8-cde2b543684f\") " pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"
Nov 04 02:18:00.833872 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:00.833835    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3d654e3-9e45-46e3-bad8-cde2b543684f-home\") pod \"simulator-w1-8cf267-kserve-f647797c8-qrfr9\" (UID: \"a3d654e3-9e45-46e3-bad8-cde2b543684f\") " pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"
Nov 04 02:18:00.835681 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:00.835662    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3d654e3-9e45-46e3-bad8-cde2b543684f-tls-certs\") pod \"simulator-w1-8cf267-kserve-f647797c8-qrfr9\" (UID: \"a3d654e3-9e45-46e3-bad8-cde2b543684f\") " pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"
Nov 04 02:18:00.835874 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:00.835856    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3d654e3-9e45-46e3-bad8-cde2b543684f-dshm\") pod \"simulator-w1-8cf267-kserve-f647797c8-qrfr9\" (UID: \"a3d654e3-9e45-46e3-bad8-cde2b543684f\") " pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"
Nov 04 02:18:00.894268 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:00.894244    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzb2w\" (UniqueName: \"kubernetes.io/projected/a3d654e3-9e45-46e3-bad8-cde2b543684f-kube-api-access-zzb2w\") pod \"simulator-w1-8cf267-kserve-f647797c8-qrfr9\" (UID: \"a3d654e3-9e45-46e3-bad8-cde2b543684f\") " pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"
Nov 04 02:18:00.977909 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:00.977854    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"
Nov 04 02:18:01.355366 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:01.355337    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"]
Nov 04 02:18:01.356858 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:18:01.356831    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda3d654e3_9e45_46e3_bad8_cde2b543684f.slice/crio-a03a780b09314534b6520a0c80a3765a81dcd85d812c5e2d371702ef2a5a9938 WatchSource:0}: Error finding container a03a780b09314534b6520a0c80a3765a81dcd85d812c5e2d371702ef2a5a9938: Status 404 returned error can't find the container with id a03a780b09314534b6520a0c80a3765a81dcd85d812c5e2d371702ef2a5a9938
Nov 04 02:18:01.434593 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:01.434566    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:02.177165 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:02.177124    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9" event={"ID":"a3d654e3-9e45-46e3-bad8-cde2b543684f","Type":"ContainerStarted","Data":"6dbcc84441e57aa2ebaa8a83e06333dfcaf2e74f3472f11e3f92c3876aa8c753"}
Nov 04 02:18:02.177165 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:02.177170    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9" event={"ID":"a3d654e3-9e45-46e3-bad8-cde2b543684f","Type":"ContainerStarted","Data":"a03a780b09314534b6520a0c80a3765a81dcd85d812c5e2d371702ef2a5a9938"}
Nov 04 02:18:02.296323 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:02.296273    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9" podStartSLOduration=2.296260946 podStartE2EDuration="2.296260946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:18:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:18:02.293020189 +0000 UTC m=+718.219242822" watchObservedRunningTime="2026-11-04 02:18:02.296260946 +0000 UTC m=+718.222483591"
Nov 04 02:18:02.978837 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:02.978802    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"
Nov 04 02:18:04.260530 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:18:04.260491    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:18:06.331048 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:06.331008    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:10.978057 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:10.978021    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"
Nov 04 02:18:10.991273 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:10.991249    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"
Nov 04 02:18:11.240598 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:11.240525    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"
Nov 04 02:18:11.318275 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:11.318238    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:14.226517 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:14.226485    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:16.145941 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:16.145912    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:21.043589 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:21.043544    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:21.228002 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:21.227968    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:24.167208 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:24.167157    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"]
Nov 04 02:18:24.172176 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:24.172154    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"
Nov 04 02:18:24.227926 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:24.227890    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"]
Nov 04 02:18:24.271840 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:24.271807    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f1779bb-f57c-4235-9ef9-baf26eafbca7-tmp-dir\") pod \"simulator-w4-e970c7-kserve-86dd96bf9d-mp78q\" (UID: \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\") " pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"
Nov 04 02:18:24.271999 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:24.271969    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmz5h\" (UniqueName: \"kubernetes.io/projected/4f1779bb-f57c-4235-9ef9-baf26eafbca7-kube-api-access-wmz5h\") pod \"simulator-w4-e970c7-kserve-86dd96bf9d-mp78q\" (UID: \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\") " pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"
Nov 04 02:18:24.272123 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:24.272105    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f1779bb-f57c-4235-9ef9-baf26eafbca7-model-cache\") pod \"simulator-w4-e970c7-kserve-86dd96bf9d-mp78q\" (UID: \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\") " pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"
Nov 04 02:18:24.272197 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:24.272147    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f1779bb-f57c-4235-9ef9-baf26eafbca7-tls-certs\") pod \"simulator-w4-e970c7-kserve-86dd96bf9d-mp78q\" (UID: \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\") " pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"
Nov 04 02:18:24.272197 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:24.272180    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f1779bb-f57c-4235-9ef9-baf26eafbca7-home\") pod \"simulator-w4-e970c7-kserve-86dd96bf9d-mp78q\" (UID: \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\") " pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"
Nov 04 02:18:24.272296 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:24.272205    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f1779bb-f57c-4235-9ef9-baf26eafbca7-dshm\") pod \"simulator-w4-e970c7-kserve-86dd96bf9d-mp78q\" (UID: \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\") " pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"
Nov 04 02:18:24.273695 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:24.273675    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-e970c7\"/\"kube-root-ca.crt\""
Nov 04 02:18:24.273842 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:24.273675    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-e970c7\"/\"openshift-service-ca.crt\""
Nov 04 02:18:24.274338 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:24.274319    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-e970c7\"/\"default-dockercfg-fsxkn\""
Nov 04 02:18:24.274430 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:24.274358    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-e970c7\"/\"simulator-w4-e970c7-kserve-self-signed-certs\""
Nov 04 02:18:24.291898 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:24.291873    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:24.373630 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:24.373591    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f1779bb-f57c-4235-9ef9-baf26eafbca7-tmp-dir\") pod \"simulator-w4-e970c7-kserve-86dd96bf9d-mp78q\" (UID: \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\") " pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"
Nov 04 02:18:24.373817 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:24.373693    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmz5h\" (UniqueName: \"kubernetes.io/projected/4f1779bb-f57c-4235-9ef9-baf26eafbca7-kube-api-access-wmz5h\") pod \"simulator-w4-e970c7-kserve-86dd96bf9d-mp78q\" (UID: \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\") " pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"
Nov 04 02:18:24.373817 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:24.373770    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f1779bb-f57c-4235-9ef9-baf26eafbca7-model-cache\") pod \"simulator-w4-e970c7-kserve-86dd96bf9d-mp78q\" (UID: \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\") " pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"
Nov 04 02:18:24.373817 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:24.373799    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f1779bb-f57c-4235-9ef9-baf26eafbca7-tls-certs\") pod \"simulator-w4-e970c7-kserve-86dd96bf9d-mp78q\" (UID: \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\") " pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"
Nov 04 02:18:24.373985 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:24.373828    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f1779bb-f57c-4235-9ef9-baf26eafbca7-home\") pod \"simulator-w4-e970c7-kserve-86dd96bf9d-mp78q\" (UID: \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\") " pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"
Nov 04 02:18:24.373985 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:24.373852    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f1779bb-f57c-4235-9ef9-baf26eafbca7-dshm\") pod \"simulator-w4-e970c7-kserve-86dd96bf9d-mp78q\" (UID: \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\") " pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"
Nov 04 02:18:24.374121 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:24.374067    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f1779bb-f57c-4235-9ef9-baf26eafbca7-tmp-dir\") pod \"simulator-w4-e970c7-kserve-86dd96bf9d-mp78q\" (UID: \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\") " pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"
Nov 04 02:18:24.374213 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:24.374191    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f1779bb-f57c-4235-9ef9-baf26eafbca7-model-cache\") pod \"simulator-w4-e970c7-kserve-86dd96bf9d-mp78q\" (UID: \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\") " pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"
Nov 04 02:18:24.374346 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:24.374325    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f1779bb-f57c-4235-9ef9-baf26eafbca7-home\") pod \"simulator-w4-e970c7-kserve-86dd96bf9d-mp78q\" (UID: \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\") " pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"
Nov 04 02:18:24.376919 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:24.376788    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f1779bb-f57c-4235-9ef9-baf26eafbca7-dshm\") pod \"simulator-w4-e970c7-kserve-86dd96bf9d-mp78q\" (UID: \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\") " pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"
Nov 04 02:18:24.377745 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:24.377718    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f1779bb-f57c-4235-9ef9-baf26eafbca7-tls-certs\") pod \"simulator-w4-e970c7-kserve-86dd96bf9d-mp78q\" (UID: \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\") " pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"
Nov 04 02:18:24.436454 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:24.436393    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmz5h\" (UniqueName: \"kubernetes.io/projected/4f1779bb-f57c-4235-9ef9-baf26eafbca7-kube-api-access-wmz5h\") pod \"simulator-w4-e970c7-kserve-86dd96bf9d-mp78q\" (UID: \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\") " pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"
Nov 04 02:18:24.486888 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:24.486861    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"
Nov 04 02:18:24.875650 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:24.875625    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"]
Nov 04 02:18:24.877942 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:18:24.877912    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f1779bb_f57c_4235_9ef9_baf26eafbca7.slice/crio-f6b925d5f4939f93d5e7b6b94d0e96c2cb271a1670078b0c06714afbe909e10a WatchSource:0}: Error finding container f6b925d5f4939f93d5e7b6b94d0e96c2cb271a1670078b0c06714afbe909e10a: Status 404 returned error can't find the container with id f6b925d5f4939f93d5e7b6b94d0e96c2cb271a1670078b0c06714afbe909e10a
Nov 04 02:18:25.321031 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:25.320991    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q" event={"ID":"4f1779bb-f57c-4235-9ef9-baf26eafbca7","Type":"ContainerStarted","Data":"c5ce85d9ddf9b9520deb795cca3cb824452406647b82421cb72293efa7b0244d"}
Nov 04 02:18:25.321031 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:25.321036    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q" event={"ID":"4f1779bb-f57c-4235-9ef9-baf26eafbca7","Type":"ContainerStarted","Data":"f6b925d5f4939f93d5e7b6b94d0e96c2cb271a1670078b0c06714afbe909e10a"}
Nov 04 02:18:25.437824 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:25.437775    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q" podStartSLOduration=1.437761601 podStartE2EDuration="1.437761601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:18:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:18:25.437533275 +0000 UTC m=+741.363755908" watchObservedRunningTime="2026-11-04 02:18:25.437761601 +0000 UTC m=+741.363984231"
Nov 04 02:18:25.486973 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:25.486945    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"
Nov 04 02:18:25.930429 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:25.930392    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:30.920983 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:30.920940    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:34.260814 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:18:34.260781    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:18:34.487656 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:34.487625    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"
Nov 04 02:18:34.501502 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:34.501465    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"
Nov 04 02:18:35.391724 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:35.391691    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"
Nov 04 02:18:35.828630 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:35.828593    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:40.728762 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:40.728727    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:45.609694 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:45.609587    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:50.529678 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:50.529648    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:52.942895 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:52.942862    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:55.438419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:55.438385    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:18:58.275053 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:18:58.275018    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:00.340874 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:00.340837    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:04.262034 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:19:04.261994    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:19:05.338474 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:05.338443    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:10.245209 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:10.245163    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:13.202881 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:13.202840    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"]
Nov 04 02:19:13.207256 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:13.207236    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"
Nov 04 02:19:13.259454 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:13.259428    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-9adedd\"/\"openshift-service-ca.crt\""
Nov 04 02:19:13.279460 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:13.279433    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfqs8\" (UniqueName: \"kubernetes.io/projected/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-kube-api-access-hfqs8\") pod \"premium-w6-9adedd-kserve-645dd5fcf-62lnl\" (UID: \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\") " pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"
Nov 04 02:19:13.279643 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:13.279468    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-tmp-dir\") pod \"premium-w6-9adedd-kserve-645dd5fcf-62lnl\" (UID: \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\") " pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"
Nov 04 02:19:13.279643 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:13.279493    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-tls-certs\") pod \"premium-w6-9adedd-kserve-645dd5fcf-62lnl\" (UID: \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\") " pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"
Nov 04 02:19:13.279774 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:13.279751    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-dshm\") pod \"premium-w6-9adedd-kserve-645dd5fcf-62lnl\" (UID: \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\") " pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"
Nov 04 02:19:13.279885 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:13.279864    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-home\") pod \"premium-w6-9adedd-kserve-645dd5fcf-62lnl\" (UID: \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\") " pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"
Nov 04 02:19:13.279968 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:13.279952    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-model-cache\") pod \"premium-w6-9adedd-kserve-645dd5fcf-62lnl\" (UID: \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\") " pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"
Nov 04 02:19:13.309166 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:13.309135    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-9adedd\"/\"premium-w6-9adedd-kserve-self-signed-certs\""
Nov 04 02:19:13.313554 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:13.313530    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-9adedd\"/\"default-dockercfg-jk7zs\""
Nov 04 02:19:13.313713 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:13.313568    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-9adedd\"/\"kube-root-ca.crt\""
Nov 04 02:19:13.316774 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:13.316753    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"]
Nov 04 02:19:13.325126 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:13.325065    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:13.381249 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:13.381212    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-model-cache\") pod \"premium-w6-9adedd-kserve-645dd5fcf-62lnl\" (UID: \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\") " pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"
Nov 04 02:19:13.381454 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:13.381314    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfqs8\" (UniqueName: \"kubernetes.io/projected/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-kube-api-access-hfqs8\") pod \"premium-w6-9adedd-kserve-645dd5fcf-62lnl\" (UID: \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\") " pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"
Nov 04 02:19:13.381454 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:13.381349    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-tmp-dir\") pod \"premium-w6-9adedd-kserve-645dd5fcf-62lnl\" (UID: \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\") " pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"
Nov 04 02:19:13.381454 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:13.381373    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-tls-certs\") pod \"premium-w6-9adedd-kserve-645dd5fcf-62lnl\" (UID: \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\") " pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"
Nov 04 02:19:13.381454 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:13.381431    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-dshm\") pod \"premium-w6-9adedd-kserve-645dd5fcf-62lnl\" (UID: \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\") " pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"
Nov 04 02:19:13.381665 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:13.381496    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-home\") pod \"premium-w6-9adedd-kserve-645dd5fcf-62lnl\" (UID: \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\") " pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"
Nov 04 02:19:13.381715 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:13.381695    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-model-cache\") pod \"premium-w6-9adedd-kserve-645dd5fcf-62lnl\" (UID: \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\") " pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"
Nov 04 02:19:13.381984 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:13.381962    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-tmp-dir\") pod \"premium-w6-9adedd-kserve-645dd5fcf-62lnl\" (UID: \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\") " pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"
Nov 04 02:19:13.382087 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:13.382056    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-home\") pod \"premium-w6-9adedd-kserve-645dd5fcf-62lnl\" (UID: \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\") " pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"
Nov 04 02:19:13.383819 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:13.383800    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-dshm\") pod \"premium-w6-9adedd-kserve-645dd5fcf-62lnl\" (UID: \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\") " pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"
Nov 04 02:19:13.384310 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:13.384295    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-tls-certs\") pod \"premium-w6-9adedd-kserve-645dd5fcf-62lnl\" (UID: \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\") " pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"
Nov 04 02:19:13.442705 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:13.442677    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfqs8\" (UniqueName: \"kubernetes.io/projected/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-kube-api-access-hfqs8\") pod \"premium-w6-9adedd-kserve-645dd5fcf-62lnl\" (UID: \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\") " pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"
Nov 04 02:19:13.519220 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:13.519189    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"
Nov 04 02:19:13.899801 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:13.899772    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"]
Nov 04 02:19:13.901425 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:19:13.901392    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod92e2ae41_9ce7_47d9_915c_bd6d70ddd2de.slice/crio-d296701a6a40726083a072ee92057bf9d93a5384120918cfa147d2509bf358c6 WatchSource:0}: Error finding container d296701a6a40726083a072ee92057bf9d93a5384120918cfa147d2509bf358c6: Status 404 returned error can't find the container with id d296701a6a40726083a072ee92057bf9d93a5384120918cfa147d2509bf358c6
Nov 04 02:19:14.628443 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:14.628398    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl" event={"ID":"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de","Type":"ContainerStarted","Data":"1f26e31f666e7675089a67a0f36f2fe72f0aae94bb59ae72facb9ac67de52e8c"}
Nov 04 02:19:14.628443 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:14.628451    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl" event={"ID":"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de","Type":"ContainerStarted","Data":"d296701a6a40726083a072ee92057bf9d93a5384120918cfa147d2509bf358c6"}
Nov 04 02:19:14.746447 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:14.746395    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl" podStartSLOduration=1.74638047 podStartE2EDuration="1.74638047s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:19:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:19:14.744487692 +0000 UTC m=+790.670710327" watchObservedRunningTime="2026-11-04 02:19:14.74638047 +0000 UTC m=+790.672603103"
Nov 04 02:19:15.225589 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:15.225556    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:15.520194 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:15.520167    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"
Nov 04 02:19:20.139955 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:20.139915    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:20.724739 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:20.724684    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:23.519727 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:23.519689    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"
Nov 04 02:19:23.532203 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:23.532174    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"
Nov 04 02:19:23.698240 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:23.698207    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"
Nov 04 02:19:25.028590 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:25.028554    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:30.030127 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:30.030080    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:32.751184 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:32.751152    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"]
Nov 04 02:19:32.755403 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:32.755383    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"
Nov 04 02:19:32.828497 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:32.828467    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"]
Nov 04 02:19:32.859443 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:32.859416    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-8cf267\"/\"premium-w1-8cf267-kserve-self-signed-certs\""
Nov 04 02:19:32.875614 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:32.875568    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:32.903776 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:32.903730    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3dba90b-2c06-4bf8-8b89-525dcada1934-home\") pod \"premium-w1-8cf267-kserve-696694b45f-w2ht9\" (UID: \"f3dba90b-2c06-4bf8-8b89-525dcada1934\") " pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"
Nov 04 02:19:32.904111 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:32.904072    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3dba90b-2c06-4bf8-8b89-525dcada1934-model-cache\") pod \"premium-w1-8cf267-kserve-696694b45f-w2ht9\" (UID: \"f3dba90b-2c06-4bf8-8b89-525dcada1934\") " pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"
Nov 04 02:19:32.904250 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:32.904235    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3dba90b-2c06-4bf8-8b89-525dcada1934-dshm\") pod \"premium-w1-8cf267-kserve-696694b45f-w2ht9\" (UID: \"f3dba90b-2c06-4bf8-8b89-525dcada1934\") " pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"
Nov 04 02:19:32.904370 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:32.904357    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3dba90b-2c06-4bf8-8b89-525dcada1934-tmp-dir\") pod \"premium-w1-8cf267-kserve-696694b45f-w2ht9\" (UID: \"f3dba90b-2c06-4bf8-8b89-525dcada1934\") " pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"
Nov 04 02:19:32.904501 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:32.904489    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3dba90b-2c06-4bf8-8b89-525dcada1934-tls-certs\") pod \"premium-w1-8cf267-kserve-696694b45f-w2ht9\" (UID: \"f3dba90b-2c06-4bf8-8b89-525dcada1934\") " pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"
Nov 04 02:19:32.904730 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:32.904718    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lfn8\" (UniqueName: \"kubernetes.io/projected/f3dba90b-2c06-4bf8-8b89-525dcada1934-kube-api-access-4lfn8\") pod \"premium-w1-8cf267-kserve-696694b45f-w2ht9\" (UID: \"f3dba90b-2c06-4bf8-8b89-525dcada1934\") " pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"
Nov 04 02:19:33.006468 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:33.006395    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lfn8\" (UniqueName: \"kubernetes.io/projected/f3dba90b-2c06-4bf8-8b89-525dcada1934-kube-api-access-4lfn8\") pod \"premium-w1-8cf267-kserve-696694b45f-w2ht9\" (UID: \"f3dba90b-2c06-4bf8-8b89-525dcada1934\") " pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"
Nov 04 02:19:33.006468 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:33.006442    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3dba90b-2c06-4bf8-8b89-525dcada1934-home\") pod \"premium-w1-8cf267-kserve-696694b45f-w2ht9\" (UID: \"f3dba90b-2c06-4bf8-8b89-525dcada1934\") " pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"
Nov 04 02:19:33.006673 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:33.006482    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3dba90b-2c06-4bf8-8b89-525dcada1934-model-cache\") pod \"premium-w1-8cf267-kserve-696694b45f-w2ht9\" (UID: \"f3dba90b-2c06-4bf8-8b89-525dcada1934\") " pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"
Nov 04 02:19:33.006673 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:33.006516    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3dba90b-2c06-4bf8-8b89-525dcada1934-dshm\") pod \"premium-w1-8cf267-kserve-696694b45f-w2ht9\" (UID: \"f3dba90b-2c06-4bf8-8b89-525dcada1934\") " pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"
Nov 04 02:19:33.006673 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:33.006555    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3dba90b-2c06-4bf8-8b89-525dcada1934-tmp-dir\") pod \"premium-w1-8cf267-kserve-696694b45f-w2ht9\" (UID: \"f3dba90b-2c06-4bf8-8b89-525dcada1934\") " pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"
Nov 04 02:19:33.006673 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:33.006601    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3dba90b-2c06-4bf8-8b89-525dcada1934-tls-certs\") pod \"premium-w1-8cf267-kserve-696694b45f-w2ht9\" (UID: \"f3dba90b-2c06-4bf8-8b89-525dcada1934\") " pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"
Nov 04 02:19:33.006967 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:33.006944    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3dba90b-2c06-4bf8-8b89-525dcada1934-home\") pod \"premium-w1-8cf267-kserve-696694b45f-w2ht9\" (UID: \"f3dba90b-2c06-4bf8-8b89-525dcada1934\") " pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"
Nov 04 02:19:33.006967 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:33.006962    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3dba90b-2c06-4bf8-8b89-525dcada1934-model-cache\") pod \"premium-w1-8cf267-kserve-696694b45f-w2ht9\" (UID: \"f3dba90b-2c06-4bf8-8b89-525dcada1934\") " pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"
Nov 04 02:19:33.007119 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:33.007071    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3dba90b-2c06-4bf8-8b89-525dcada1934-tmp-dir\") pod \"premium-w1-8cf267-kserve-696694b45f-w2ht9\" (UID: \"f3dba90b-2c06-4bf8-8b89-525dcada1934\") " pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"
Nov 04 02:19:33.008887 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:33.008867    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3dba90b-2c06-4bf8-8b89-525dcada1934-dshm\") pod \"premium-w1-8cf267-kserve-696694b45f-w2ht9\" (UID: \"f3dba90b-2c06-4bf8-8b89-525dcada1934\") " pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"
Nov 04 02:19:33.009203 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:33.009180    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3dba90b-2c06-4bf8-8b89-525dcada1934-tls-certs\") pod \"premium-w1-8cf267-kserve-696694b45f-w2ht9\" (UID: \"f3dba90b-2c06-4bf8-8b89-525dcada1934\") " pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"
Nov 04 02:19:33.070754 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:33.070723    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lfn8\" (UniqueName: \"kubernetes.io/projected/f3dba90b-2c06-4bf8-8b89-525dcada1934-kube-api-access-4lfn8\") pod \"premium-w1-8cf267-kserve-696694b45f-w2ht9\" (UID: \"f3dba90b-2c06-4bf8-8b89-525dcada1934\") " pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"
Nov 04 02:19:33.071548 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:33.071504    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"
Nov 04 02:19:33.461127 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:33.460940    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"]
Nov 04 02:19:33.461278 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:19:33.461179    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3dba90b_2c06_4bf8_8b89_525dcada1934.slice/crio-90d38905a2da8d6aa8038bb875433a8135fd5ea2db492e57ee25e4b7c9e7f997 WatchSource:0}: Error finding container 90d38905a2da8d6aa8038bb875433a8135fd5ea2db492e57ee25e4b7c9e7f997: Status 404 returned error can't find the container with id 90d38905a2da8d6aa8038bb875433a8135fd5ea2db492e57ee25e4b7c9e7f997
Nov 04 02:19:33.751544 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:33.751466    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9" event={"ID":"f3dba90b-2c06-4bf8-8b89-525dcada1934","Type":"ContainerStarted","Data":"01fe666ae8b8268988789eb9377bb8068480369b1fb83ce813a0877e1bbb0b15"}
Nov 04 02:19:33.751544 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:33.751516    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9" event={"ID":"f3dba90b-2c06-4bf8-8b89-525dcada1934","Type":"ContainerStarted","Data":"90d38905a2da8d6aa8038bb875433a8135fd5ea2db492e57ee25e4b7c9e7f997"}
Nov 04 02:19:33.869248 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:33.869191    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9" podStartSLOduration=1.869178496 podStartE2EDuration="1.869178496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:19:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:19:33.867322199 +0000 UTC m=+809.793544833" watchObservedRunningTime="2026-11-04 02:19:33.869178496 +0000 UTC m=+809.795401129"
Nov 04 02:19:34.071870 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:34.071820    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"
Nov 04 02:19:34.264134 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:19:34.263028    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:19:34.918863 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:34.918804    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:39.753795 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:39.753767    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:43.072188 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:43.072148    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"
Nov 04 02:19:43.085783 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:43.085754    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"
Nov 04 02:19:43.828337 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:43.828307    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"
Nov 04 02:19:44.621515 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:44.621484    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:49.508281 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:49.508252    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:54.027760 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:54.027728    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:54.522374 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:54.522343    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:58.617470 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:58.617437    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:19:59.635884 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:19:59.635847    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:04.261263 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:20:04.261223    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:20:04.522709 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:04.522619    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:07.271132 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:07.271077    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"]
Nov 04 02:20:07.279701 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:07.279674    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"
Nov 04 02:20:07.344847 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:07.344810    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"]
Nov 04 02:20:07.377012 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:07.376974    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e246b7c-de7c-42be-b07c-0d95bd019886-tmp-dir\") pod \"premium-w4-e970c7-kserve-7864d99f67-5mqhd\" (UID: \"6e246b7c-de7c-42be-b07c-0d95bd019886\") " pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"
Nov 04 02:20:07.377381 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:07.377358    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e246b7c-de7c-42be-b07c-0d95bd019886-model-cache\") pod \"premium-w4-e970c7-kserve-7864d99f67-5mqhd\" (UID: \"6e246b7c-de7c-42be-b07c-0d95bd019886\") " pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"
Nov 04 02:20:07.377468 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:07.377402    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e246b7c-de7c-42be-b07c-0d95bd019886-dshm\") pod \"premium-w4-e970c7-kserve-7864d99f67-5mqhd\" (UID: \"6e246b7c-de7c-42be-b07c-0d95bd019886\") " pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"
Nov 04 02:20:07.377468 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:07.377421    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e246b7c-de7c-42be-b07c-0d95bd019886-tls-certs\") pod \"premium-w4-e970c7-kserve-7864d99f67-5mqhd\" (UID: \"6e246b7c-de7c-42be-b07c-0d95bd019886\") " pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"
Nov 04 02:20:07.377468 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:07.377446    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e246b7c-de7c-42be-b07c-0d95bd019886-home\") pod \"premium-w4-e970c7-kserve-7864d99f67-5mqhd\" (UID: \"6e246b7c-de7c-42be-b07c-0d95bd019886\") " pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"
Nov 04 02:20:07.377629 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:07.377499    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whb5j\" (UniqueName: \"kubernetes.io/projected/6e246b7c-de7c-42be-b07c-0d95bd019886-kube-api-access-whb5j\") pod \"premium-w4-e970c7-kserve-7864d99f67-5mqhd\" (UID: \"6e246b7c-de7c-42be-b07c-0d95bd019886\") " pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"
Nov 04 02:20:07.379771 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:07.379749    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-e970c7\"/\"premium-w4-e970c7-kserve-self-signed-certs\""
Nov 04 02:20:07.396110 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:07.395993    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:07.478114 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:07.478061    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e246b7c-de7c-42be-b07c-0d95bd019886-model-cache\") pod \"premium-w4-e970c7-kserve-7864d99f67-5mqhd\" (UID: \"6e246b7c-de7c-42be-b07c-0d95bd019886\") " pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"
Nov 04 02:20:07.478316 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:07.478126    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e246b7c-de7c-42be-b07c-0d95bd019886-dshm\") pod \"premium-w4-e970c7-kserve-7864d99f67-5mqhd\" (UID: \"6e246b7c-de7c-42be-b07c-0d95bd019886\") " pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"
Nov 04 02:20:07.478316 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:07.478157    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e246b7c-de7c-42be-b07c-0d95bd019886-tls-certs\") pod \"premium-w4-e970c7-kserve-7864d99f67-5mqhd\" (UID: \"6e246b7c-de7c-42be-b07c-0d95bd019886\") " pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"
Nov 04 02:20:07.478316 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:07.478189    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e246b7c-de7c-42be-b07c-0d95bd019886-home\") pod \"premium-w4-e970c7-kserve-7864d99f67-5mqhd\" (UID: \"6e246b7c-de7c-42be-b07c-0d95bd019886\") " pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"
Nov 04 02:20:07.478316 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:07.478257    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whb5j\" (UniqueName: \"kubernetes.io/projected/6e246b7c-de7c-42be-b07c-0d95bd019886-kube-api-access-whb5j\") pod \"premium-w4-e970c7-kserve-7864d99f67-5mqhd\" (UID: \"6e246b7c-de7c-42be-b07c-0d95bd019886\") " pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"
Nov 04 02:20:07.478545 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:07.478396    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e246b7c-de7c-42be-b07c-0d95bd019886-tmp-dir\") pod \"premium-w4-e970c7-kserve-7864d99f67-5mqhd\" (UID: \"6e246b7c-de7c-42be-b07c-0d95bd019886\") " pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"
Nov 04 02:20:07.478545 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:07.478513    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e246b7c-de7c-42be-b07c-0d95bd019886-model-cache\") pod \"premium-w4-e970c7-kserve-7864d99f67-5mqhd\" (UID: \"6e246b7c-de7c-42be-b07c-0d95bd019886\") " pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"
Nov 04 02:20:07.478846 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:07.478754    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e246b7c-de7c-42be-b07c-0d95bd019886-tmp-dir\") pod \"premium-w4-e970c7-kserve-7864d99f67-5mqhd\" (UID: \"6e246b7c-de7c-42be-b07c-0d95bd019886\") " pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"
Nov 04 02:20:07.478846 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:07.478789    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e246b7c-de7c-42be-b07c-0d95bd019886-home\") pod \"premium-w4-e970c7-kserve-7864d99f67-5mqhd\" (UID: \"6e246b7c-de7c-42be-b07c-0d95bd019886\") " pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"
Nov 04 02:20:07.480824 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:07.480789    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e246b7c-de7c-42be-b07c-0d95bd019886-dshm\") pod \"premium-w4-e970c7-kserve-7864d99f67-5mqhd\" (UID: \"6e246b7c-de7c-42be-b07c-0d95bd019886\") " pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"
Nov 04 02:20:07.482355 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:07.482317    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e246b7c-de7c-42be-b07c-0d95bd019886-tls-certs\") pod \"premium-w4-e970c7-kserve-7864d99f67-5mqhd\" (UID: \"6e246b7c-de7c-42be-b07c-0d95bd019886\") " pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"
Nov 04 02:20:07.539997 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:07.539914    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whb5j\" (UniqueName: \"kubernetes.io/projected/6e246b7c-de7c-42be-b07c-0d95bd019886-kube-api-access-whb5j\") pod \"premium-w4-e970c7-kserve-7864d99f67-5mqhd\" (UID: \"6e246b7c-de7c-42be-b07c-0d95bd019886\") " pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"
Nov 04 02:20:07.614518 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:07.614485    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"
Nov 04 02:20:08.002118 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:08.002065    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"]
Nov 04 02:20:08.005087 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:20:08.005057    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e246b7c_de7c_42be_b07c_0d95bd019886.slice/crio-970526d7148762856c950606a7b4e91f6a8dbed95983876745014c56c5df21fc WatchSource:0}: Error finding container 970526d7148762856c950606a7b4e91f6a8dbed95983876745014c56c5df21fc: Status 404 returned error can't find the container with id 970526d7148762856c950606a7b4e91f6a8dbed95983876745014c56c5df21fc
Nov 04 02:20:08.990771 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:08.990732    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd" event={"ID":"6e246b7c-de7c-42be-b07c-0d95bd019886","Type":"ContainerStarted","Data":"485db69a1937cfe290cceaf9d8ce62d5087bdccada7e691eb27607efe6317484"}
Nov 04 02:20:08.990771 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:08.990778    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd" event={"ID":"6e246b7c-de7c-42be-b07c-0d95bd019886","Type":"ContainerStarted","Data":"970526d7148762856c950606a7b4e91f6a8dbed95983876745014c56c5df21fc"}
Nov 04 02:20:09.109088 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:09.109036    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd" podStartSLOduration=2.109022968 podStartE2EDuration="2.109022968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:20:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:20:09.106583335 +0000 UTC m=+845.032805968" watchObservedRunningTime="2026-11-04 02:20:09.109022968 +0000 UTC m=+845.035245600"
Nov 04 02:20:09.517736 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:09.517700    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:09.614647 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:09.614607    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"
Nov 04 02:20:11.446729 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:11.446695    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:11.629000 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:11.628968    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:14.397163 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:14.397051    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:17.615155 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:17.615122    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"
Nov 04 02:20:17.627737 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:17.627712    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"
Nov 04 02:20:18.059601 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:18.059570    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"
Nov 04 02:20:19.214197 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:19.214153    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:24.134699 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:24.134640    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:29.027835 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:29.027758    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:31.804682 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:31.804638    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:33.899063 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:33.899019    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:34.241942 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:34.241866    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:34.263020 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:20:34.262983    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:20:35.806260 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:35.806228    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:38.936241 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:38.936197    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:43.753899 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:43.753863    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:48.634257 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:48.634220    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:52.729861 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:52.729823    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"]
Nov 04 02:20:52.730346 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:52.730037    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb" podUID="0093557b-c9ae-4bd8-b9ea-019f62cd69d1" containerName="istio-proxy" containerID="cri-o://de716654f44b991a9176f2acd678e85223b08068263740534ab1e06fa10c3dab" gracePeriod=30
Nov 04 02:20:53.536078 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:53.536042    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:58.003654 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.003589    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:20:58.040761 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.040735    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-kube-api-access-xmk67\" (UniqueName: \"kubernetes.io/projected/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-kube-api-access-xmk67\") pod \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") "
Nov 04 02:20:58.040913 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.040793    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istio-podinfo\") pod \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") "
Nov 04 02:20:58.040913 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.040816    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istiod-ca-cert\") pod \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") "
Nov 04 02:20:58.040913 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.040902    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-workload-socket\") pod \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") "
Nov 04 02:20:58.041180 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.040938    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istio-envoy\") pod \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") "
Nov 04 02:20:58.041180 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.040976    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-credential-socket\") pod \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") "
Nov 04 02:20:58.041180 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.041052    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istio-data\") pod \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") "
Nov 04 02:20:58.041180 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.041141    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istio-token\" (UniqueName: \"kubernetes.io/projected/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istio-token\") pod \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") "
Nov 04 02:20:58.041398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.041181    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-workload-certs\") pod \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\" (UID: \"0093557b-c9ae-4bd8-b9ea-019f62cd69d1\") "
Nov 04 02:20:58.041398 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.041283    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-workload-socket" pod "0093557b-c9ae-4bd8-b9ea-019f62cd69d1" (UID: "0093557b-c9ae-4bd8-b9ea-019f62cd69d1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:20:58.041574 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.041469    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istiod-ca-cert" pod "0093557b-c9ae-4bd8-b9ea-019f62cd69d1" (UID: "0093557b-c9ae-4bd8-b9ea-019f62cd69d1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:20:58.041574 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.041515    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-credential-socket" pod "0093557b-c9ae-4bd8-b9ea-019f62cd69d1" (UID: "0093557b-c9ae-4bd8-b9ea-019f62cd69d1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:20:58.041767 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.041743    4969 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istiod-ca-cert\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:58.041847 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.041774    4969 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-workload-socket\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:58.041847 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.041790    4969 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-credential-socket\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:58.042049 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.042012    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istio-data" pod "0093557b-c9ae-4bd8-b9ea-019f62cd69d1" (UID: "0093557b-c9ae-4bd8-b9ea-019f62cd69d1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:20:58.042194 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.042134    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-workload-certs" pod "0093557b-c9ae-4bd8-b9ea-019f62cd69d1" (UID: "0093557b-c9ae-4bd8-b9ea-019f62cd69d1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:20:58.043261 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.043240    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istio-podinfo" pod "0093557b-c9ae-4bd8-b9ea-019f62cd69d1" (UID: "0093557b-c9ae-4bd8-b9ea-019f62cd69d1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:20:58.043466 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.043448    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-kube-api-access-xmk67" pod "0093557b-c9ae-4bd8-b9ea-019f62cd69d1" (UID: "0093557b-c9ae-4bd8-b9ea-019f62cd69d1"). InnerVolumeSpecName "kube-api-access-xmk67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:20:58.043875 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.043850    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istio-envoy" pod "0093557b-c9ae-4bd8-b9ea-019f62cd69d1" (UID: "0093557b-c9ae-4bd8-b9ea-019f62cd69d1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:20:58.044055 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.044036    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istio-token" pod "0093557b-c9ae-4bd8-b9ea-019f62cd69d1" (UID: "0093557b-c9ae-4bd8-b9ea-019f62cd69d1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:20:58.143405 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.143379    4969 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istio-data\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:58.143405 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.143405    4969 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istio-token\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:58.143641 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.143417    4969 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-workload-certs\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:58.143641 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.143427    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmk67\" (UniqueName: \"kubernetes.io/projected/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-kube-api-access-xmk67\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:58.143641 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.143437    4969 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istio-podinfo\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:58.143641 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.143445    4969 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0093557b-c9ae-4bd8-b9ea-019f62cd69d1-istio-envoy\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:20:58.323065 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.323032    4969 generic.go:356] "Generic (PLEG): container finished" podID="0093557b-c9ae-4bd8-b9ea-019f62cd69d1" containerID="de716654f44b991a9176f2acd678e85223b08068263740534ab1e06fa10c3dab" exitCode=0
Nov 04 02:20:58.323261 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.323105    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"
Nov 04 02:20:58.323261 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.323127    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb" event={"ID":"0093557b-c9ae-4bd8-b9ea-019f62cd69d1","Type":"ContainerDied","Data":"de716654f44b991a9176f2acd678e85223b08068263740534ab1e06fa10c3dab"}
Nov 04 02:20:58.323261 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.323204    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb" event={"ID":"0093557b-c9ae-4bd8-b9ea-019f62cd69d1","Type":"ContainerDied","Data":"2bec1e6b44e8d7308fb430374ce081edab503ab90b6cba91b7e9df51a3f0c998"}
Nov 04 02:20:58.323261 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.323226    4969 scope.go:122] "RemoveContainer" containerID="de716654f44b991a9176f2acd678e85223b08068263740534ab1e06fa10c3dab"
Nov 04 02:20:58.333467 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.333450    4969 scope.go:122] "RemoveContainer" containerID="de716654f44b991a9176f2acd678e85223b08068263740534ab1e06fa10c3dab"
Nov 04 02:20:58.333765 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:20:58.333738    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de716654f44b991a9176f2acd678e85223b08068263740534ab1e06fa10c3dab\": container with ID starting with de716654f44b991a9176f2acd678e85223b08068263740534ab1e06fa10c3dab not found: ID does not exist" containerID="de716654f44b991a9176f2acd678e85223b08068263740534ab1e06fa10c3dab"
Nov 04 02:20:58.333897 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.333780    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de716654f44b991a9176f2acd678e85223b08068263740534ab1e06fa10c3dab"} err="failed to get container status \"de716654f44b991a9176f2acd678e85223b08068263740534ab1e06fa10c3dab\": rpc error: code = NotFound desc = could not find container \"de716654f44b991a9176f2acd678e85223b08068263740534ab1e06fa10c3dab\": container with ID starting with de716654f44b991a9176f2acd678e85223b08068263740534ab1e06fa10c3dab not found: ID does not exist"
Nov 04 02:20:58.492145 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.492076    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"]
Nov 04 02:20:58.494873 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.494851    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-qnjkb"]
Nov 04 02:20:58.525219 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.525189    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:20:58.975204 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:20:58.975173    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:00.203396 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:00.203364    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0093557b-c9ae-4bd8-b9ea-019f62cd69d1" path="/var/lib/kubelet/pods/0093557b-c9ae-4bd8-b9ea-019f62cd69d1/volumes"
Nov 04 02:21:03.228335 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:03.228304    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:03.445805 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:03.445770    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:03.654806 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:03.654769    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:04.266417 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:21:04.266383    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:21:04.317151 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:04.317131    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jcqmp_16f00163-6ec3-4c10-81e4-c8a501e15e82/console-operator/2.log"
Nov 04 02:21:04.318845 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:04.318822    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jcqmp_16f00163-6ec3-4c10-81e4-c8a501e15e82/console-operator/2.log"
Nov 04 02:21:08.375044 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:08.375013    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:09.826665 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:09.826633    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:13.227128 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:13.227033    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:18.129773 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:18.129737    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:21.130392 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:21.130354    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:21.376801 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:21.376769    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:22.109569 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:22.109533    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:22.820684 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:22.820650    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"]
Nov 04 02:21:22.821289 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:22.821274    4969 cpu_manager.go:414] "containerMap: removing container" podUID="0093557b-c9ae-4bd8-b9ea-019f62cd69d1" containerName="istio-proxy"
Nov 04 02:21:22.821345 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:22.821292    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0093557b-c9ae-4bd8-b9ea-019f62cd69d1" containerName="istio-proxy"
Nov 04 02:21:22.821438 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:22.821429    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="0093557b-c9ae-4bd8-b9ea-019f62cd69d1" containerName="istio-proxy"
Nov 04 02:21:22.824947 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:22.824925    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:22.877440 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:22.877418    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-5a27ba-openshift-default-dockercfg-r2gf6\""
Nov 04 02:21:22.929939 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:22.929913    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"]
Nov 04 02:21:23.010662 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.010632    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istio-data\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:23.010835 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.010673    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sts8\" (UniqueName: \"kubernetes.io/projected/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-kube-api-access-6sts8\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:23.010835 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.010747    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-workload-socket\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:23.010835 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.010776    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istio-envoy\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:23.010835 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.010795    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istio-podinfo\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:23.010835 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.010833    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istiod-ca-cert\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:23.011006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.010978    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istio-token\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:23.011043 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.011007    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-credential-socket\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:23.011086 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.011074    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-workload-certs\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:23.055428 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.055394    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:23.112619 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.112527    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istio-data\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:23.112619 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.112569    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sts8\" (UniqueName: \"kubernetes.io/projected/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-kube-api-access-6sts8\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:23.112619 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.112605    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-workload-socket\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:23.112619 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.112623    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istio-envoy\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:23.112963 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.112640    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istio-podinfo\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:23.112963 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.112678    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istiod-ca-cert\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:23.112963 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.112723    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istio-token\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:23.112963 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.112754    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-credential-socket\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:23.112963 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.112802    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-workload-certs\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:23.113261 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.113217    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-credential-socket\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:23.113320 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.113262    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-workload-socket\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:23.113320 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.113275    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istio-data\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:23.113542 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.113522    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-workload-certs\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:23.113631 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.113611    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istiod-ca-cert\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:23.115122 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.115072    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istio-envoy\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:23.115327 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.115311    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istio-podinfo\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:23.175700 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.175663    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istio-token\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:23.175829 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.175734    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sts8\" (UniqueName: \"kubernetes.io/projected/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-kube-api-access-6sts8\") pod \"e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") " pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:23.310732 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.310695    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:23.440394 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.440303    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:23.871733 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.871705    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"]
Nov 04 02:21:23.872704 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:21:23.872672    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1c402e7_f796_4c4a_af73_b9f07a9c98cc.slice/crio-50c30f29f81fb77e93fe2ba6c2bdf2a481ca9a372c609a11b63159955050d796 WatchSource:0}: Error finding container 50c30f29f81fb77e93fe2ba6c2bdf2a481ca9a372c609a11b63159955050d796: Status 404 returned error can't find the container with id 50c30f29f81fb77e93fe2ba6c2bdf2a481ca9a372c609a11b63159955050d796
Nov 04 02:21:23.875043 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.875008    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:21:23.875151 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.875072    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:21:23.875151 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:23.875119    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:21:24.492895 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:24.492864    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2" event={"ID":"f1c402e7-f796-4c4a-af73-b9f07a9c98cc","Type":"ContainerStarted","Data":"c1f6c2343d35d7654764112ad8652f83e338c1a7688cd6264edd92209a517f23"}
Nov 04 02:21:24.493072 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:24.492914    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2" event={"ID":"f1c402e7-f796-4c4a-af73-b9f07a9c98cc","Type":"ContainerStarted","Data":"50c30f29f81fb77e93fe2ba6c2bdf2a481ca9a372c609a11b63159955050d796"}
Nov 04 02:21:24.586493 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:24.586456    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:24.690366 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:24.690300    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2" podStartSLOduration=2.690281845 podStartE2EDuration="2.690281845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:21:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:21:24.682659218 +0000 UTC m=+920.608881865" watchObservedRunningTime="2026-11-04 02:21:24.690281845 +0000 UTC m=+920.616504479"
Nov 04 02:21:25.156882 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.156848    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"]
Nov 04 02:21:25.181389 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.181355    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:25.222598 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.222562    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"]
Nov 04 02:21:25.239128 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.233739    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-e34b54-openshift-default-dockercfg-vpxtz\""
Nov 04 02:21:25.340949 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.340912    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istio-data\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:25.341194 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.341078    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-workload-socket\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:25.341282 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.341261    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istio-podinfo\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:25.341337 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.341291    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz5c2\" (UniqueName: \"kubernetes.io/projected/6d2b50e4-af78-4a23-8a66-000bfd766bb3-kube-api-access-pz5c2\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:25.341393 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.341365    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-workload-certs\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:25.341440 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.341394    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istiod-ca-cert\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:25.341533 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.341508    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istio-envoy\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:25.341603 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.341539    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istio-token\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:25.341603 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.341568    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-credential-socket\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:25.440824 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.440737    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:25.442020 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.441993    4969 patch_prober.go:29] interesting pod/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 04 02:21:25.442171 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.442047    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2" podUID="f1c402e7-f796-4c4a-af73-b9f07a9c98cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 04 02:21:25.442448 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.442427    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istio-podinfo\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:25.442486 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.442459    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz5c2\" (UniqueName: \"kubernetes.io/projected/6d2b50e4-af78-4a23-8a66-000bfd766bb3-kube-api-access-pz5c2\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:25.442606 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.442587    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-workload-certs\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:25.442660 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.442639    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istiod-ca-cert\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:25.442714 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.442699    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istio-envoy\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:25.442769 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.442726    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istio-token\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:25.442820 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.442765    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-credential-socket\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:25.442820 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.442813    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istio-data\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:25.442935 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.442883    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-workload-socket\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:25.443055 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.443034    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-workload-certs\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:25.443218 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.443193    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istio-data\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:25.443277 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.443237    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-workload-socket\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:25.443430 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.443407    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istiod-ca-cert\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:25.443521 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.443458    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-credential-socket\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:25.445212 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.445186    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istio-podinfo\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:25.445581 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.445559    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istio-envoy\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:25.503175 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.503144    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istio-token\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:25.503818 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.503788    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz5c2\" (UniqueName: \"kubernetes.io/projected/6d2b50e4-af78-4a23-8a66-000bfd766bb3-kube-api-access-pz5c2\") pod \"e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") " pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:25.801608 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:25.801577    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:26.283207 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:26.283172    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"]
Nov 04 02:21:26.301710 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:21:26.301682    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d2b50e4_af78_4a23_8a66_000bfd766bb3.slice/crio-a168ced9b0a6e828ba1e1abfd8510d878bb69321343fbcb70d5fefee3acedf66 WatchSource:0}: Error finding container a168ced9b0a6e828ba1e1abfd8510d878bb69321343fbcb70d5fefee3acedf66: Status 404 returned error can't find the container with id a168ced9b0a6e828ba1e1abfd8510d878bb69321343fbcb70d5fefee3acedf66
Nov 04 02:21:26.304069 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:26.304040    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:21:26.304171 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:26.304120    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:21:26.304212 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:26.304151    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:21:26.440788 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:26.440753    4969 patch_prober.go:29] interesting pod/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 04 02:21:26.440956 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:26.440813    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2" podUID="f1c402e7-f796-4c4a-af73-b9f07a9c98cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 04 02:21:26.509879 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:26.509841    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4" event={"ID":"6d2b50e4-af78-4a23-8a66-000bfd766bb3","Type":"ContainerStarted","Data":"487fc85a35ad834004f00cd7b0657ef139e09bee09aa4d46997f923d8e816594"}
Nov 04 02:21:26.510062 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:26.509899    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4" event={"ID":"6d2b50e4-af78-4a23-8a66-000bfd766bb3","Type":"ContainerStarted","Data":"a168ced9b0a6e828ba1e1abfd8510d878bb69321343fbcb70d5fefee3acedf66"}
Nov 04 02:21:26.642079 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:26.641982    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4" podStartSLOduration=1.641969444 podStartE2EDuration="1.641969444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:21:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:21:26.639467144 +0000 UTC m=+922.565689779" watchObservedRunningTime="2026-11-04 02:21:26.641969444 +0000 UTC m=+922.568192077"
Nov 04 02:21:26.801867 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:26.801830    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:27.441160 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:27.441126    4969 patch_prober.go:29] interesting pod/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 04 02:21:27.441547 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:27.441182    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2" podUID="f1c402e7-f796-4c4a-af73-b9f07a9c98cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 04 02:21:27.808231 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:27.806441    4969 patch_prober.go:29] interesting pod/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 04 02:21:27.808231 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:27.806509    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4" podUID="6d2b50e4-af78-4a23-8a66-000bfd766bb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 04 02:21:28.208299 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:28.203447    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:28.396135 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:28.394560    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:28.441448 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:28.441418    4969 patch_prober.go:29] interesting pod/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 04 02:21:28.441873 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:28.441850    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2" podUID="f1c402e7-f796-4c4a-af73-b9f07a9c98cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 04 02:21:28.806113 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:28.805558    4969 patch_prober.go:29] interesting pod/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 04 02:21:28.806113 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:28.805654    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4" podUID="6d2b50e4-af78-4a23-8a66-000bfd766bb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 04 02:21:29.041113 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:29.034565    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:29.445250 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:29.445208    4969 patch_prober.go:29] interesting pod/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 04 02:21:29.445852 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:29.445805    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2" podUID="f1c402e7-f796-4c4a-af73-b9f07a9c98cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 04 02:21:29.723223 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:29.723086    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:29.812293 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:29.811409    4969 patch_prober.go:29] interesting pod/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 04 02:21:29.812293 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:29.811480    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4" podUID="6d2b50e4-af78-4a23-8a66-000bfd766bb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 04 02:21:30.304335 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:30.304304    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:30.440606 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:30.440528    4969 patch_prober.go:29] interesting pod/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 04 02:21:30.440606 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:30.440595    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2" podUID="f1c402e7-f796-4c4a-af73-b9f07a9c98cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 04 02:21:30.803713 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:30.803678    4969 patch_prober.go:29] interesting pod/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 04 02:21:30.804303 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:30.804271    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4" podUID="6d2b50e4-af78-4a23-8a66-000bfd766bb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 04 02:21:31.445489 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:31.445455    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:31.445837 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:31.445816    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:31.446546 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:31.446528    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:21:31.802778 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:31.802746    4969 patch_prober.go:29] interesting pod/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 04 02:21:31.803498 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:31.803031    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4" podUID="6d2b50e4-af78-4a23-8a66-000bfd766bb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 04 02:21:32.110765 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:32.110683    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:32.802776 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:32.802730    4969 patch_prober.go:29] interesting pod/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 04 02:21:32.802982 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:32.802790    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4" podUID="6d2b50e4-af78-4a23-8a66-000bfd766bb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 04 02:21:33.336037 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:33.336001    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:33.806791 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:33.806764    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:33.807136 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:33.807109    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:33.807930 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:33.807911    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:21:33.919967 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:33.919936    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"]
Nov 04 02:21:33.924794 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:33.924759    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"]
Nov 04 02:21:34.034015 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:34.033983    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:34.277248 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:21:34.277208    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:21:35.347445 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:35.347415    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:35.837258 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:35.837224    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:37.016500 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:37.016468    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"]
Nov 04 02:21:37.065937 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:37.065909    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"]
Nov 04 02:21:37.208921 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:37.208890    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:38.224326 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:38.224294    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:40.302835 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:40.302806    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:41.566017 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:41.565983    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:42.231075 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:42.231045    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:43.189964 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:43.189923    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:44.178100 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:44.178060    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd"]
Nov 04 02:21:44.288942 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:44.288906    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj"]
Nov 04 02:21:44.627380 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:44.627353    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:44.810363 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:44.810331    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:46.754205 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:46.754171    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:47.686402 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:47.686366    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:48.452779 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:48.452734    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:48.749873 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:48.749781    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:48.935266 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:48.935233    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:48.949454 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:48.949420    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9" podUID="f3dba90b-2c06-4bf8-8b89-525dcada1934" containerName="main" containerID="cri-o://01fe666ae8b8268988789eb9377bb8068480369b1fb83ce813a0877e1bbb0b15" gracePeriod=45
Nov 04 02:21:48.954254 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:48.954230    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9" podUID="a3d654e3-9e45-46e3-bad8-cde2b543684f" containerName="main" containerID="cri-o://6dbcc84441e57aa2ebaa8a83e06333dfcaf2e74f3472f11e3f92c3876aa8c753" gracePeriod=45
Nov 04 02:21:49.180054 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.179987    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd" podUID="440b09dc-d2a6-4879-8b71-e189dd565c27" containerName="payload-processing" containerID="cri-o://8371a6c2aace7aff4430f2a356f6ddfc0591242c349230f0e343cbfa95560d44" gracePeriod=25
Nov 04 02:21:49.290261 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.290210    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj" podUID="43d4ca4c-da17-4f49-898e-d08e931bc93b" containerName="payload-pre-processing" containerID="cri-o://b107c0546c13db77241de79405a565ae385d1c4abe27b87bda64e7ebca588907" gracePeriod=25
Nov 04 02:21:49.378949 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.375963    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"
Nov 04 02:21:49.378949 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.378447    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"
Nov 04 02:21:49.445925 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.445895    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd"
Nov 04 02:21:49.557275 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.556971    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3d654e3-9e45-46e3-bad8-cde2b543684f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3d654e3-9e45-46e3-bad8-cde2b543684f-model-cache\") pod \"a3d654e3-9e45-46e3-bad8-cde2b543684f\" (UID: \"a3d654e3-9e45-46e3-bad8-cde2b543684f\") "
Nov 04 02:21:49.557275 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.557040    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a3d654e3-9e45-46e3-bad8-cde2b543684f-kube-api-access-zzb2w\" (UniqueName: \"kubernetes.io/projected/a3d654e3-9e45-46e3-bad8-cde2b543684f-kube-api-access-zzb2w\") pod \"a3d654e3-9e45-46e3-bad8-cde2b543684f\" (UID: \"a3d654e3-9e45-46e3-bad8-cde2b543684f\") "
Nov 04 02:21:49.557275 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.557106    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/440b09dc-d2a6-4879-8b71-e189dd565c27-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/440b09dc-d2a6-4879-8b71-e189dd565c27-plugins-config-volume\") pod \"440b09dc-d2a6-4879-8b71-e189dd565c27\" (UID: \"440b09dc-d2a6-4879-8b71-e189dd565c27\") "
Nov 04 02:21:49.557275 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.557191    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3dba90b-2c06-4bf8-8b89-525dcada1934-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3dba90b-2c06-4bf8-8b89-525dcada1934-dshm\") pod \"f3dba90b-2c06-4bf8-8b89-525dcada1934\" (UID: \"f3dba90b-2c06-4bf8-8b89-525dcada1934\") "
Nov 04 02:21:49.557275 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.557244    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3dba90b-2c06-4bf8-8b89-525dcada1934-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3dba90b-2c06-4bf8-8b89-525dcada1934-model-cache\") pod \"f3dba90b-2c06-4bf8-8b89-525dcada1934\" (UID: \"f3dba90b-2c06-4bf8-8b89-525dcada1934\") "
Nov 04 02:21:49.557757 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.557291    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f3dba90b-2c06-4bf8-8b89-525dcada1934-tls-certs\" (UniqueName: \"kubernetes.io/secret/f3dba90b-2c06-4bf8-8b89-525dcada1934-tls-certs\") pod \"f3dba90b-2c06-4bf8-8b89-525dcada1934\" (UID: \"f3dba90b-2c06-4bf8-8b89-525dcada1934\") "
Nov 04 02:21:49.557757 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.557325    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/440b09dc-d2a6-4879-8b71-e189dd565c27-kube-api-access-h6k7h\" (UniqueName: \"kubernetes.io/projected/440b09dc-d2a6-4879-8b71-e189dd565c27-kube-api-access-h6k7h\") pod \"440b09dc-d2a6-4879-8b71-e189dd565c27\" (UID: \"440b09dc-d2a6-4879-8b71-e189dd565c27\") "
Nov 04 02:21:49.557757 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.557381    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3dba90b-2c06-4bf8-8b89-525dcada1934-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3dba90b-2c06-4bf8-8b89-525dcada1934-tmp-dir\") pod \"f3dba90b-2c06-4bf8-8b89-525dcada1934\" (UID: \"f3dba90b-2c06-4bf8-8b89-525dcada1934\") "
Nov 04 02:21:49.557757 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.557434    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3d654e3-9e45-46e3-bad8-cde2b543684f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3d654e3-9e45-46e3-bad8-cde2b543684f-dshm\") pod \"a3d654e3-9e45-46e3-bad8-cde2b543684f\" (UID: \"a3d654e3-9e45-46e3-bad8-cde2b543684f\") "
Nov 04 02:21:49.557757 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.557480    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3d654e3-9e45-46e3-bad8-cde2b543684f-home\" (UniqueName: \"kubernetes.io/empty-dir/a3d654e3-9e45-46e3-bad8-cde2b543684f-home\") pod \"a3d654e3-9e45-46e3-bad8-cde2b543684f\" (UID: \"a3d654e3-9e45-46e3-bad8-cde2b543684f\") "
Nov 04 02:21:49.557757 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.557527    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a3d654e3-9e45-46e3-bad8-cde2b543684f-tls-certs\" (UniqueName: \"kubernetes.io/secret/a3d654e3-9e45-46e3-bad8-cde2b543684f-tls-certs\") pod \"a3d654e3-9e45-46e3-bad8-cde2b543684f\" (UID: \"a3d654e3-9e45-46e3-bad8-cde2b543684f\") "
Nov 04 02:21:49.557757 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.557555    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f3dba90b-2c06-4bf8-8b89-525dcada1934-kube-api-access-4lfn8\" (UniqueName: \"kubernetes.io/projected/f3dba90b-2c06-4bf8-8b89-525dcada1934-kube-api-access-4lfn8\") pod \"f3dba90b-2c06-4bf8-8b89-525dcada1934\" (UID: \"f3dba90b-2c06-4bf8-8b89-525dcada1934\") "
Nov 04 02:21:49.557757 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.557624    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3d654e3-9e45-46e3-bad8-cde2b543684f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3d654e3-9e45-46e3-bad8-cde2b543684f-tmp-dir\") pod \"a3d654e3-9e45-46e3-bad8-cde2b543684f\" (UID: \"a3d654e3-9e45-46e3-bad8-cde2b543684f\") "
Nov 04 02:21:49.557757 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.557655    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3dba90b-2c06-4bf8-8b89-525dcada1934-home\" (UniqueName: \"kubernetes.io/empty-dir/f3dba90b-2c06-4bf8-8b89-525dcada1934-home\") pod \"f3dba90b-2c06-4bf8-8b89-525dcada1934\" (UID: \"f3dba90b-2c06-4bf8-8b89-525dcada1934\") "
Nov 04 02:21:49.558841 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.558812    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3dba90b-2c06-4bf8-8b89-525dcada1934-home" pod "f3dba90b-2c06-4bf8-8b89-525dcada1934" (UID: "f3dba90b-2c06-4bf8-8b89-525dcada1934"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:49.559031 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.559009    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3dba90b-2c06-4bf8-8b89-525dcada1934-tmp-dir" pod "f3dba90b-2c06-4bf8-8b89-525dcada1934" (UID: "f3dba90b-2c06-4bf8-8b89-525dcada1934"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:49.560452 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.560260    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3d654e3-9e45-46e3-bad8-cde2b543684f-home" pod "a3d654e3-9e45-46e3-bad8-cde2b543684f" (UID: "a3d654e3-9e45-46e3-bad8-cde2b543684f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:49.560452 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.560291    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3d654e3-9e45-46e3-bad8-cde2b543684f-tmp-dir" pod "a3d654e3-9e45-46e3-bad8-cde2b543684f" (UID: "a3d654e3-9e45-46e3-bad8-cde2b543684f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:49.560452 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.560290    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3dba90b-2c06-4bf8-8b89-525dcada1934-model-cache" pod "f3dba90b-2c06-4bf8-8b89-525dcada1934" (UID: "f3dba90b-2c06-4bf8-8b89-525dcada1934"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:49.560669 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.560483    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3d654e3-9e45-46e3-bad8-cde2b543684f-model-cache" pod "a3d654e3-9e45-46e3-bad8-cde2b543684f" (UID: "a3d654e3-9e45-46e3-bad8-cde2b543684f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:49.560926 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.560901    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/440b09dc-d2a6-4879-8b71-e189dd565c27-plugins-config-volume" pod "440b09dc-d2a6-4879-8b71-e189dd565c27" (UID: "440b09dc-d2a6-4879-8b71-e189dd565c27"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:21:49.563135 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.562063    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/440b09dc-d2a6-4879-8b71-e189dd565c27-kube-api-access-h6k7h" pod "440b09dc-d2a6-4879-8b71-e189dd565c27" (UID: "440b09dc-d2a6-4879-8b71-e189dd565c27"). InnerVolumeSpecName "kube-api-access-h6k7h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:49.563135 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.562794    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3d654e3-9e45-46e3-bad8-cde2b543684f-dshm" pod "a3d654e3-9e45-46e3-bad8-cde2b543684f" (UID: "a3d654e3-9e45-46e3-bad8-cde2b543684f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:49.563964 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.563923    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3dba90b-2c06-4bf8-8b89-525dcada1934-dshm" pod "f3dba90b-2c06-4bf8-8b89-525dcada1934" (UID: "f3dba90b-2c06-4bf8-8b89-525dcada1934"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:49.564230 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.564198    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3d654e3-9e45-46e3-bad8-cde2b543684f-tls-certs" pod "a3d654e3-9e45-46e3-bad8-cde2b543684f" (UID: "a3d654e3-9e45-46e3-bad8-cde2b543684f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:21:49.564509 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.564489    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3d654e3-9e45-46e3-bad8-cde2b543684f-kube-api-access-zzb2w" pod "a3d654e3-9e45-46e3-bad8-cde2b543684f" (UID: "a3d654e3-9e45-46e3-bad8-cde2b543684f"). InnerVolumeSpecName "kube-api-access-zzb2w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:49.564680 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.564663    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3dba90b-2c06-4bf8-8b89-525dcada1934-kube-api-access-4lfn8" pod "f3dba90b-2c06-4bf8-8b89-525dcada1934" (UID: "f3dba90b-2c06-4bf8-8b89-525dcada1934"). InnerVolumeSpecName "kube-api-access-4lfn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:49.565603 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.565569    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3dba90b-2c06-4bf8-8b89-525dcada1934-tls-certs" pod "f3dba90b-2c06-4bf8-8b89-525dcada1934" (UID: "f3dba90b-2c06-4bf8-8b89-525dcada1934"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:21:49.591839 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.591815    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj"
Nov 04 02:21:49.659527 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.659492    4969 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3dba90b-2c06-4bf8-8b89-525dcada1934-model-cache\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:49.659527 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.659526    4969 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3dba90b-2c06-4bf8-8b89-525dcada1934-tls-certs\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:49.659723 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.659538    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6k7h\" (UniqueName: \"kubernetes.io/projected/440b09dc-d2a6-4879-8b71-e189dd565c27-kube-api-access-h6k7h\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:49.659723 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.659547    4969 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3dba90b-2c06-4bf8-8b89-525dcada1934-tmp-dir\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:49.659723 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.659567    4969 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3d654e3-9e45-46e3-bad8-cde2b543684f-dshm\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:49.659723 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.659574    4969 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3d654e3-9e45-46e3-bad8-cde2b543684f-home\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:49.659723 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.659584    4969 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3d654e3-9e45-46e3-bad8-cde2b543684f-tls-certs\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:49.659723 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.659592    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lfn8\" (UniqueName: \"kubernetes.io/projected/f3dba90b-2c06-4bf8-8b89-525dcada1934-kube-api-access-4lfn8\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:49.659723 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.659600    4969 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3d654e3-9e45-46e3-bad8-cde2b543684f-tmp-dir\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:49.659723 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.659607    4969 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3dba90b-2c06-4bf8-8b89-525dcada1934-home\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:49.659723 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.659615    4969 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3d654e3-9e45-46e3-bad8-cde2b543684f-model-cache\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:49.659723 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.659626    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzb2w\" (UniqueName: \"kubernetes.io/projected/a3d654e3-9e45-46e3-bad8-cde2b543684f-kube-api-access-zzb2w\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:49.659723 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.659634    4969 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/440b09dc-d2a6-4879-8b71-e189dd565c27-plugins-config-volume\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:49.659723 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.659645    4969 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3dba90b-2c06-4bf8-8b89-525dcada1934-dshm\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:49.742661 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.742628    4969 generic.go:356] "Generic (PLEG): container finished" podID="440b09dc-d2a6-4879-8b71-e189dd565c27" containerID="8371a6c2aace7aff4430f2a356f6ddfc0591242c349230f0e343cbfa95560d44" exitCode=0
Nov 04 02:21:49.757587 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.742721    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd" event={"ID":"440b09dc-d2a6-4879-8b71-e189dd565c27","Type":"ContainerDied","Data":"8371a6c2aace7aff4430f2a356f6ddfc0591242c349230f0e343cbfa95560d44"}
Nov 04 02:21:49.757587 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.742874    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd" event={"ID":"440b09dc-d2a6-4879-8b71-e189dd565c27","Type":"ContainerDied","Data":"02afb26fe63053fa97efe8165eaf09c69f075f971de6820c685e6f6bc1c349ed"}
Nov 04 02:21:49.757587 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.742908    4969 scope.go:122] "RemoveContainer" containerID="8371a6c2aace7aff4430f2a356f6ddfc0591242c349230f0e343cbfa95560d44"
Nov 04 02:21:49.757587 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.742930    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd"
Nov 04 02:21:49.757587 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.754505    4969 generic.go:356] "Generic (PLEG): container finished" podID="a3d654e3-9e45-46e3-bad8-cde2b543684f" containerID="6dbcc84441e57aa2ebaa8a83e06333dfcaf2e74f3472f11e3f92c3876aa8c753" exitCode=0
Nov 04 02:21:49.757587 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.754560    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"
Nov 04 02:21:49.757587 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.754596    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9" event={"ID":"a3d654e3-9e45-46e3-bad8-cde2b543684f","Type":"ContainerDied","Data":"6dbcc84441e57aa2ebaa8a83e06333dfcaf2e74f3472f11e3f92c3876aa8c753"}
Nov 04 02:21:49.757587 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.754630    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9" event={"ID":"a3d654e3-9e45-46e3-bad8-cde2b543684f","Type":"ContainerDied","Data":"a03a780b09314534b6520a0c80a3765a81dcd85d812c5e2d371702ef2a5a9938"}
Nov 04 02:21:49.757587 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.757561    4969 generic.go:356] "Generic (PLEG): container finished" podID="f3dba90b-2c06-4bf8-8b89-525dcada1934" containerID="01fe666ae8b8268988789eb9377bb8068480369b1fb83ce813a0877e1bbb0b15" exitCode=0
Nov 04 02:21:49.758391 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.757602    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"
Nov 04 02:21:49.758391 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.757828    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9" event={"ID":"f3dba90b-2c06-4bf8-8b89-525dcada1934","Type":"ContainerDied","Data":"01fe666ae8b8268988789eb9377bb8068480369b1fb83ce813a0877e1bbb0b15"}
Nov 04 02:21:49.758391 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.757862    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9" event={"ID":"f3dba90b-2c06-4bf8-8b89-525dcada1934","Type":"ContainerDied","Data":"90d38905a2da8d6aa8038bb875433a8135fd5ea2db492e57ee25e4b7c9e7f997"}
Nov 04 02:21:49.760245 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.760217    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/43d4ca4c-da17-4f49-898e-d08e931bc93b-kube-api-access-jqq24\" (UniqueName: \"kubernetes.io/projected/43d4ca4c-da17-4f49-898e-d08e931bc93b-kube-api-access-jqq24\") pod \"43d4ca4c-da17-4f49-898e-d08e931bc93b\" (UID: \"43d4ca4c-da17-4f49-898e-d08e931bc93b\") "
Nov 04 02:21:49.760494 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.760472    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/43d4ca4c-da17-4f49-898e-d08e931bc93b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43d4ca4c-da17-4f49-898e-d08e931bc93b-plugins-config-volume\") pod \"43d4ca4c-da17-4f49-898e-d08e931bc93b\" (UID: \"43d4ca4c-da17-4f49-898e-d08e931bc93b\") "
Nov 04 02:21:49.761252 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.761228    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43d4ca4c-da17-4f49-898e-d08e931bc93b-plugins-config-volume" pod "43d4ca4c-da17-4f49-898e-d08e931bc93b" (UID: "43d4ca4c-da17-4f49-898e-d08e931bc93b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:21:49.774283 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.774252    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43d4ca4c-da17-4f49-898e-d08e931bc93b-kube-api-access-jqq24" pod "43d4ca4c-da17-4f49-898e-d08e931bc93b" (UID: "43d4ca4c-da17-4f49-898e-d08e931bc93b"). InnerVolumeSpecName "kube-api-access-jqq24". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:49.777263 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.774811    4969 generic.go:356] "Generic (PLEG): container finished" podID="43d4ca4c-da17-4f49-898e-d08e931bc93b" containerID="b107c0546c13db77241de79405a565ae385d1c4abe27b87bda64e7ebca588907" exitCode=0
Nov 04 02:21:49.777263 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.774919    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj" event={"ID":"43d4ca4c-da17-4f49-898e-d08e931bc93b","Type":"ContainerDied","Data":"b107c0546c13db77241de79405a565ae385d1c4abe27b87bda64e7ebca588907"}
Nov 04 02:21:49.777263 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.774963    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj" event={"ID":"43d4ca4c-da17-4f49-898e-d08e931bc93b","Type":"ContainerDied","Data":"862235b41507605a3339185d3801c94a202fb0b03b3d46f6d46aef688be5d011"}
Nov 04 02:21:49.777263 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.775035    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj"
Nov 04 02:21:49.786154 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.786084    4969 scope.go:122] "RemoveContainer" containerID="8371a6c2aace7aff4430f2a356f6ddfc0591242c349230f0e343cbfa95560d44"
Nov 04 02:21:49.793194 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:21:49.791804    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8371a6c2aace7aff4430f2a356f6ddfc0591242c349230f0e343cbfa95560d44\": container with ID starting with 8371a6c2aace7aff4430f2a356f6ddfc0591242c349230f0e343cbfa95560d44 not found: ID does not exist" containerID="8371a6c2aace7aff4430f2a356f6ddfc0591242c349230f0e343cbfa95560d44"
Nov 04 02:21:49.793194 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.791837    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8371a6c2aace7aff4430f2a356f6ddfc0591242c349230f0e343cbfa95560d44"} err="failed to get container status \"8371a6c2aace7aff4430f2a356f6ddfc0591242c349230f0e343cbfa95560d44\": rpc error: code = NotFound desc = could not find container \"8371a6c2aace7aff4430f2a356f6ddfc0591242c349230f0e343cbfa95560d44\": container with ID starting with 8371a6c2aace7aff4430f2a356f6ddfc0591242c349230f0e343cbfa95560d44 not found: ID does not exist"
Nov 04 02:21:49.793194 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.791863    4969 scope.go:122] "RemoveContainer" containerID="6dbcc84441e57aa2ebaa8a83e06333dfcaf2e74f3472f11e3f92c3876aa8c753"
Nov 04 02:21:49.809133 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.809109    4969 scope.go:122] "RemoveContainer" containerID="6dbcc84441e57aa2ebaa8a83e06333dfcaf2e74f3472f11e3f92c3876aa8c753"
Nov 04 02:21:49.809457 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:21:49.809432    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6dbcc84441e57aa2ebaa8a83e06333dfcaf2e74f3472f11e3f92c3876aa8c753\": container with ID starting with 6dbcc84441e57aa2ebaa8a83e06333dfcaf2e74f3472f11e3f92c3876aa8c753 not found: ID does not exist" containerID="6dbcc84441e57aa2ebaa8a83e06333dfcaf2e74f3472f11e3f92c3876aa8c753"
Nov 04 02:21:49.809530 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.809479    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6dbcc84441e57aa2ebaa8a83e06333dfcaf2e74f3472f11e3f92c3876aa8c753"} err="failed to get container status \"6dbcc84441e57aa2ebaa8a83e06333dfcaf2e74f3472f11e3f92c3876aa8c753\": rpc error: code = NotFound desc = could not find container \"6dbcc84441e57aa2ebaa8a83e06333dfcaf2e74f3472f11e3f92c3876aa8c753\": container with ID starting with 6dbcc84441e57aa2ebaa8a83e06333dfcaf2e74f3472f11e3f92c3876aa8c753 not found: ID does not exist"
Nov 04 02:21:49.809530 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.809504    4969 scope.go:122] "RemoveContainer" containerID="01fe666ae8b8268988789eb9377bb8068480369b1fb83ce813a0877e1bbb0b15"
Nov 04 02:21:49.818880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.818860    4969 scope.go:122] "RemoveContainer" containerID="01fe666ae8b8268988789eb9377bb8068480369b1fb83ce813a0877e1bbb0b15"
Nov 04 02:21:49.819236 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:21:49.819205    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01fe666ae8b8268988789eb9377bb8068480369b1fb83ce813a0877e1bbb0b15\": container with ID starting with 01fe666ae8b8268988789eb9377bb8068480369b1fb83ce813a0877e1bbb0b15 not found: ID does not exist" containerID="01fe666ae8b8268988789eb9377bb8068480369b1fb83ce813a0877e1bbb0b15"
Nov 04 02:21:49.819323 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.819236    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01fe666ae8b8268988789eb9377bb8068480369b1fb83ce813a0877e1bbb0b15"} err="failed to get container status \"01fe666ae8b8268988789eb9377bb8068480369b1fb83ce813a0877e1bbb0b15\": rpc error: code = NotFound desc = could not find container \"01fe666ae8b8268988789eb9377bb8068480369b1fb83ce813a0877e1bbb0b15\": container with ID starting with 01fe666ae8b8268988789eb9377bb8068480369b1fb83ce813a0877e1bbb0b15 not found: ID does not exist"
Nov 04 02:21:49.819323 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.819259    4969 scope.go:122] "RemoveContainer" containerID="b107c0546c13db77241de79405a565ae385d1c4abe27b87bda64e7ebca588907"
Nov 04 02:21:49.828205 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.828188    4969 scope.go:122] "RemoveContainer" containerID="b107c0546c13db77241de79405a565ae385d1c4abe27b87bda64e7ebca588907"
Nov 04 02:21:49.828502 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:21:49.828485    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b107c0546c13db77241de79405a565ae385d1c4abe27b87bda64e7ebca588907\": container with ID starting with b107c0546c13db77241de79405a565ae385d1c4abe27b87bda64e7ebca588907 not found: ID does not exist" containerID="b107c0546c13db77241de79405a565ae385d1c4abe27b87bda64e7ebca588907"
Nov 04 02:21:49.828566 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.828508    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b107c0546c13db77241de79405a565ae385d1c4abe27b87bda64e7ebca588907"} err="failed to get container status \"b107c0546c13db77241de79405a565ae385d1c4abe27b87bda64e7ebca588907\": rpc error: code = NotFound desc = could not find container \"b107c0546c13db77241de79405a565ae385d1c4abe27b87bda64e7ebca588907\": container with ID starting with b107c0546c13db77241de79405a565ae385d1c4abe27b87bda64e7ebca588907 not found: ID does not exist"
Nov 04 02:21:49.862028 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.861769    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqq24\" (UniqueName: \"kubernetes.io/projected/43d4ca4c-da17-4f49-898e-d08e931bc93b-kube-api-access-jqq24\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:49.862235 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.862212    4969 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43d4ca4c-da17-4f49-898e-d08e931bc93b-plugins-config-volume\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:49.981848 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.981817    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd"]
Nov 04 02:21:49.988117 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:49.988076    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-9b0f0d-654788b768-qnckd"]
Nov 04 02:21:50.106700 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:50.106612    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj"]
Nov 04 02:21:50.125751 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:50.125722    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-9b0f0d-7445d9784c-58wcj"]
Nov 04 02:21:50.205375 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:50.205333    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="43d4ca4c-da17-4f49-898e-d08e931bc93b" path="/var/lib/kubelet/pods/43d4ca4c-da17-4f49-898e-d08e931bc93b/volumes"
Nov 04 02:21:50.205786 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:50.205768    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="440b09dc-d2a6-4879-8b71-e189dd565c27" path="/var/lib/kubelet/pods/440b09dc-d2a6-4879-8b71-e189dd565c27/volumes"
Nov 04 02:21:50.230444 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:50.230403    4969 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9" podUID="a3d654e3-9e45-46e3-bad8-cde2b543684f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:21:50.263039 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:50.263006    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"]
Nov 04 02:21:50.274580 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:50.274535    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-8cf267/premium-w1-8cf267-kserve-696694b45f-w2ht9"]
Nov 04 02:21:50.382598 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:50.382513    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"]
Nov 04 02:21:50.386370 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:50.386342    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-8cf267/simulator-w1-8cf267-kserve-f647797c8-qrfr9"]
Nov 04 02:21:50.759944 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:50.759914    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:51.238018 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:51.237937    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc"]
Nov 04 02:21:51.645800 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:51.645765    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:51.826793 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:51.826759    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:52.037375 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.037334    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd" podUID="6e246b7c-de7c-42be-b07c-0d95bd019886" containerName="main" containerID="cri-o://485db69a1937cfe290cceaf9d8ce62d5087bdccada7e691eb27607efe6317484" gracePeriod=45
Nov 04 02:21:52.048060 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.048029    4969 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd" podUID="6e246b7c-de7c-42be-b07c-0d95bd019886" containerName="main" probeResult="failure" output="Get \"https://10.132.0.77:8000/ready\": dial tcp 10.132.0.77:8000: connect: connection refused"
Nov 04 02:21:52.086263 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.086233    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q" podUID="4f1779bb-f57c-4235-9ef9-baf26eafbca7" containerName="main" containerID="cri-o://c5ce85d9ddf9b9520deb795cca3cb824452406647b82421cb72293efa7b0244d" gracePeriod=45
Nov 04 02:21:52.205373 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.205347    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a3d654e3-9e45-46e3-bad8-cde2b543684f" path="/var/lib/kubelet/pods/a3d654e3-9e45-46e3-bad8-cde2b543684f/volumes"
Nov 04 02:21:52.205891 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.205867    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f3dba90b-2c06-4bf8-8b89-525dcada1934" path="/var/lib/kubelet/pods/f3dba90b-2c06-4bf8-8b89-525dcada1934/volumes"
Nov 04 02:21:52.370240 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.370218    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"
Nov 04 02:21:52.403363 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.403340    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"
Nov 04 02:21:52.409410 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.409385    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6e246b7c-de7c-42be-b07c-0d95bd019886-tls-certs\" (UniqueName: \"kubernetes.io/secret/6e246b7c-de7c-42be-b07c-0d95bd019886-tls-certs\") pod \"6e246b7c-de7c-42be-b07c-0d95bd019886\" (UID: \"6e246b7c-de7c-42be-b07c-0d95bd019886\") "
Nov 04 02:21:52.409520 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.409429    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e246b7c-de7c-42be-b07c-0d95bd019886-kube-api-access-whb5j\" (UniqueName: \"kubernetes.io/projected/6e246b7c-de7c-42be-b07c-0d95bd019886-kube-api-access-whb5j\") pod \"6e246b7c-de7c-42be-b07c-0d95bd019886\" (UID: \"6e246b7c-de7c-42be-b07c-0d95bd019886\") "
Nov 04 02:21:52.409520 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.409474    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e246b7c-de7c-42be-b07c-0d95bd019886-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e246b7c-de7c-42be-b07c-0d95bd019886-dshm\") pod \"6e246b7c-de7c-42be-b07c-0d95bd019886\" (UID: \"6e246b7c-de7c-42be-b07c-0d95bd019886\") "
Nov 04 02:21:52.409520 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.409504    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e246b7c-de7c-42be-b07c-0d95bd019886-home\" (UniqueName: \"kubernetes.io/empty-dir/6e246b7c-de7c-42be-b07c-0d95bd019886-home\") pod \"6e246b7c-de7c-42be-b07c-0d95bd019886\" (UID: \"6e246b7c-de7c-42be-b07c-0d95bd019886\") "
Nov 04 02:21:52.409661 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.409538    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e246b7c-de7c-42be-b07c-0d95bd019886-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e246b7c-de7c-42be-b07c-0d95bd019886-tmp-dir\") pod \"6e246b7c-de7c-42be-b07c-0d95bd019886\" (UID: \"6e246b7c-de7c-42be-b07c-0d95bd019886\") "
Nov 04 02:21:52.409661 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.409566    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e246b7c-de7c-42be-b07c-0d95bd019886-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e246b7c-de7c-42be-b07c-0d95bd019886-model-cache\") pod \"6e246b7c-de7c-42be-b07c-0d95bd019886\" (UID: \"6e246b7c-de7c-42be-b07c-0d95bd019886\") "
Nov 04 02:21:52.409868 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.409841    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e246b7c-de7c-42be-b07c-0d95bd019886-tmp-dir" pod "6e246b7c-de7c-42be-b07c-0d95bd019886" (UID: "6e246b7c-de7c-42be-b07c-0d95bd019886"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:52.409932 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.409866    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e246b7c-de7c-42be-b07c-0d95bd019886-model-cache" pod "6e246b7c-de7c-42be-b07c-0d95bd019886" (UID: "6e246b7c-de7c-42be-b07c-0d95bd019886"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:52.409932 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.409896    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e246b7c-de7c-42be-b07c-0d95bd019886-home" pod "6e246b7c-de7c-42be-b07c-0d95bd019886" (UID: "6e246b7c-de7c-42be-b07c-0d95bd019886"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:52.410076 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.410060    4969 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6e246b7c-de7c-42be-b07c-0d95bd019886-home\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:52.410144 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.410081    4969 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e246b7c-de7c-42be-b07c-0d95bd019886-tmp-dir\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:52.410144 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.410124    4969 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6e246b7c-de7c-42be-b07c-0d95bd019886-model-cache\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:52.411567 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.411542    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e246b7c-de7c-42be-b07c-0d95bd019886-dshm" pod "6e246b7c-de7c-42be-b07c-0d95bd019886" (UID: "6e246b7c-de7c-42be-b07c-0d95bd019886"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:52.411977 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.411958    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e246b7c-de7c-42be-b07c-0d95bd019886-tls-certs" pod "6e246b7c-de7c-42be-b07c-0d95bd019886" (UID: "6e246b7c-de7c-42be-b07c-0d95bd019886"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:21:52.412018 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.411987    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e246b7c-de7c-42be-b07c-0d95bd019886-kube-api-access-whb5j" pod "6e246b7c-de7c-42be-b07c-0d95bd019886" (UID: "6e246b7c-de7c-42be-b07c-0d95bd019886"). InnerVolumeSpecName "kube-api-access-whb5j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:52.510924 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.510892    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f1779bb-f57c-4235-9ef9-baf26eafbca7-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f1779bb-f57c-4235-9ef9-baf26eafbca7-model-cache\") pod \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\" (UID: \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\") "
Nov 04 02:21:52.511085 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.510953    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4f1779bb-f57c-4235-9ef9-baf26eafbca7-kube-api-access-wmz5h\" (UniqueName: \"kubernetes.io/projected/4f1779bb-f57c-4235-9ef9-baf26eafbca7-kube-api-access-wmz5h\") pod \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\" (UID: \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\") "
Nov 04 02:21:52.511085 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.510998    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4f1779bb-f57c-4235-9ef9-baf26eafbca7-tls-certs\" (UniqueName: \"kubernetes.io/secret/4f1779bb-f57c-4235-9ef9-baf26eafbca7-tls-certs\") pod \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\" (UID: \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\") "
Nov 04 02:21:52.511085 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.511054    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f1779bb-f57c-4235-9ef9-baf26eafbca7-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f1779bb-f57c-4235-9ef9-baf26eafbca7-tmp-dir\") pod \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\" (UID: \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\") "
Nov 04 02:21:52.511267 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.511120    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f1779bb-f57c-4235-9ef9-baf26eafbca7-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f1779bb-f57c-4235-9ef9-baf26eafbca7-dshm\") pod \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\" (UID: \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\") "
Nov 04 02:21:52.511267 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.511192    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f1779bb-f57c-4235-9ef9-baf26eafbca7-model-cache" pod "4f1779bb-f57c-4235-9ef9-baf26eafbca7" (UID: "4f1779bb-f57c-4235-9ef9-baf26eafbca7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:52.511267 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.511257    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f1779bb-f57c-4235-9ef9-baf26eafbca7-home\" (UniqueName: \"kubernetes.io/empty-dir/4f1779bb-f57c-4235-9ef9-baf26eafbca7-home\") pod \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\" (UID: \"4f1779bb-f57c-4235-9ef9-baf26eafbca7\") "
Nov 04 02:21:52.511507 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.511476    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f1779bb-f57c-4235-9ef9-baf26eafbca7-tmp-dir" pod "4f1779bb-f57c-4235-9ef9-baf26eafbca7" (UID: "4f1779bb-f57c-4235-9ef9-baf26eafbca7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:52.511747 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.511723    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f1779bb-f57c-4235-9ef9-baf26eafbca7-home" pod "4f1779bb-f57c-4235-9ef9-baf26eafbca7" (UID: "4f1779bb-f57c-4235-9ef9-baf26eafbca7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:52.511839 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.511730    4969 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f1779bb-f57c-4235-9ef9-baf26eafbca7-model-cache\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:52.511839 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.511774    4969 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e246b7c-de7c-42be-b07c-0d95bd019886-tls-certs\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:52.511839 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.511791    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whb5j\" (UniqueName: \"kubernetes.io/projected/6e246b7c-de7c-42be-b07c-0d95bd019886-kube-api-access-whb5j\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:52.511839 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.511807    4969 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f1779bb-f57c-4235-9ef9-baf26eafbca7-tmp-dir\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:52.511839 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.511820    4969 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6e246b7c-de7c-42be-b07c-0d95bd019886-dshm\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:52.513332 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.513309    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f1779bb-f57c-4235-9ef9-baf26eafbca7-dshm" pod "4f1779bb-f57c-4235-9ef9-baf26eafbca7" (UID: "4f1779bb-f57c-4235-9ef9-baf26eafbca7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:21:52.513521 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.513500    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f1779bb-f57c-4235-9ef9-baf26eafbca7-kube-api-access-wmz5h" pod "4f1779bb-f57c-4235-9ef9-baf26eafbca7" (UID: "4f1779bb-f57c-4235-9ef9-baf26eafbca7"). InnerVolumeSpecName "kube-api-access-wmz5h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:52.513698 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.513674    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f1779bb-f57c-4235-9ef9-baf26eafbca7-tls-certs" pod "4f1779bb-f57c-4235-9ef9-baf26eafbca7" (UID: "4f1779bb-f57c-4235-9ef9-baf26eafbca7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:21:52.612794 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.612666    4969 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f1779bb-f57c-4235-9ef9-baf26eafbca7-tls-certs\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:52.612794 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.612697    4969 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f1779bb-f57c-4235-9ef9-baf26eafbca7-dshm\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:52.612794 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.612711    4969 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f1779bb-f57c-4235-9ef9-baf26eafbca7-home\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:52.612794 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.612725    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmz5h\" (UniqueName: \"kubernetes.io/projected/4f1779bb-f57c-4235-9ef9-baf26eafbca7-kube-api-access-wmz5h\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:52.835528 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.835486    4969 generic.go:356] "Generic (PLEG): container finished" podID="6e246b7c-de7c-42be-b07c-0d95bd019886" containerID="485db69a1937cfe290cceaf9d8ce62d5087bdccada7e691eb27607efe6317484" exitCode=0
Nov 04 02:21:52.836137 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.835580    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd" event={"ID":"6e246b7c-de7c-42be-b07c-0d95bd019886","Type":"ContainerDied","Data":"485db69a1937cfe290cceaf9d8ce62d5087bdccada7e691eb27607efe6317484"}
Nov 04 02:21:52.836266 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.836190    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd" event={"ID":"6e246b7c-de7c-42be-b07c-0d95bd019886","Type":"ContainerDied","Data":"970526d7148762856c950606a7b4e91f6a8dbed95983876745014c56c5df21fc"}
Nov 04 02:21:52.836266 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.835597    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"
Nov 04 02:21:52.836266 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.836226    4969 scope.go:122] "RemoveContainer" containerID="485db69a1937cfe290cceaf9d8ce62d5087bdccada7e691eb27607efe6317484"
Nov 04 02:21:52.840248 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.840221    4969 generic.go:356] "Generic (PLEG): container finished" podID="4f1779bb-f57c-4235-9ef9-baf26eafbca7" containerID="c5ce85d9ddf9b9520deb795cca3cb824452406647b82421cb72293efa7b0244d" exitCode=0
Nov 04 02:21:52.840440 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.840419    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q" event={"ID":"4f1779bb-f57c-4235-9ef9-baf26eafbca7","Type":"ContainerDied","Data":"c5ce85d9ddf9b9520deb795cca3cb824452406647b82421cb72293efa7b0244d"}
Nov 04 02:21:52.840569 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.840555    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q" event={"ID":"4f1779bb-f57c-4235-9ef9-baf26eafbca7","Type":"ContainerDied","Data":"f6b925d5f4939f93d5e7b6b94d0e96c2cb271a1670078b0c06714afbe909e10a"}
Nov 04 02:21:52.840842 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.840826    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"
Nov 04 02:21:52.853286 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.853245    4969 scope.go:122] "RemoveContainer" containerID="485db69a1937cfe290cceaf9d8ce62d5087bdccada7e691eb27607efe6317484"
Nov 04 02:21:52.854235 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:21:52.854202    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"485db69a1937cfe290cceaf9d8ce62d5087bdccada7e691eb27607efe6317484\": container with ID starting with 485db69a1937cfe290cceaf9d8ce62d5087bdccada7e691eb27607efe6317484 not found: ID does not exist" containerID="485db69a1937cfe290cceaf9d8ce62d5087bdccada7e691eb27607efe6317484"
Nov 04 02:21:52.854354 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.854243    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"485db69a1937cfe290cceaf9d8ce62d5087bdccada7e691eb27607efe6317484"} err="failed to get container status \"485db69a1937cfe290cceaf9d8ce62d5087bdccada7e691eb27607efe6317484\": rpc error: code = NotFound desc = could not find container \"485db69a1937cfe290cceaf9d8ce62d5087bdccada7e691eb27607efe6317484\": container with ID starting with 485db69a1937cfe290cceaf9d8ce62d5087bdccada7e691eb27607efe6317484 not found: ID does not exist"
Nov 04 02:21:52.854354 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.854267    4969 scope.go:122] "RemoveContainer" containerID="c5ce85d9ddf9b9520deb795cca3cb824452406647b82421cb72293efa7b0244d"
Nov 04 02:21:52.866235 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.866216    4969 scope.go:122] "RemoveContainer" containerID="c5ce85d9ddf9b9520deb795cca3cb824452406647b82421cb72293efa7b0244d"
Nov 04 02:21:52.866504 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:21:52.866482    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5ce85d9ddf9b9520deb795cca3cb824452406647b82421cb72293efa7b0244d\": container with ID starting with c5ce85d9ddf9b9520deb795cca3cb824452406647b82421cb72293efa7b0244d not found: ID does not exist" containerID="c5ce85d9ddf9b9520deb795cca3cb824452406647b82421cb72293efa7b0244d"
Nov 04 02:21:52.866564 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:52.866513    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5ce85d9ddf9b9520deb795cca3cb824452406647b82421cb72293efa7b0244d"} err="failed to get container status \"c5ce85d9ddf9b9520deb795cca3cb824452406647b82421cb72293efa7b0244d\": rpc error: code = NotFound desc = could not find container \"c5ce85d9ddf9b9520deb795cca3cb824452406647b82421cb72293efa7b0244d\": container with ID starting with c5ce85d9ddf9b9520deb795cca3cb824452406647b82421cb72293efa7b0244d not found: ID does not exist"
Nov 04 02:21:53.003985 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:53.003948    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"]
Nov 04 02:21:53.052722 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:53.052691    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q"]
Nov 04 02:21:53.114827 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:53.114791    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"]
Nov 04 02:21:53.117923 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:53.117863    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-e970c7/premium-w4-e970c7-kserve-7864d99f67-5mqhd"]
Nov 04 02:21:53.381046 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:53.380942    4969 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e970c7/simulator-w4-e970c7-kserve-86dd96bf9d-mp78q" podUID="4f1779bb-f57c-4235-9ef9-baf26eafbca7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": context deadline exceeded"
Nov 04 02:21:53.651318 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:53.651230    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:53.702115 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:53.702058    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl"]
Nov 04 02:21:53.816185 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:53.816154    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc"]
Nov 04 02:21:54.101069 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:54.101017    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:54.204825 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:54.204795    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4f1779bb-f57c-4235-9ef9-baf26eafbca7" path="/var/lib/kubelet/pods/4f1779bb-f57c-4235-9ef9-baf26eafbca7/volumes"
Nov 04 02:21:54.205239 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:54.205224    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6e246b7c-de7c-42be-b07c-0d95bd019886" path="/var/lib/kubelet/pods/6e246b7c-de7c-42be-b07c-0d95bd019886/volumes"
Nov 04 02:21:54.280949 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:54.280912    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:54.456025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:54.455926    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:54.641356 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:54.641315    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:55.172255 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:55.172223    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:55.347631 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:55.347595    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:56.238556 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:56.238508    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc" podUID="756a3073-fe07-4efa-9469-3862ab9350d4" containerName="payload-processing" containerID="cri-o://557bc90183d721cc7f94097a91683ad14397aede4c52cd541be0b45f2307ae4a" gracePeriod=25
Nov 04 02:21:56.566109 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:56.566068    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:56.758317 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:56.758277    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:57.612927 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:57.612890    4969 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc" podUID="756a3073-fe07-4efa-9469-3862ab9350d4" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:21:58.702991 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:58.702951    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl" podUID="6fc4dede-0532-4749-8cf2-44d868cd64c0" containerName="payload-processing" containerID="cri-o://5f69836c0421bf2b5d89d47c02103fdb43828ebdcd749a992b777d527ec913ee" gracePeriod=25
Nov 04 02:21:58.816877 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:58.816818    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc" podUID="85d5905f-412e-4e4b-8553-9adb4a72afea" containerName="payload-pre-processing" containerID="cri-o://450fa8d893d7e7bb5ec085d8b6b11d05e4c2e9bbf7ecb6eea2891070da55e042" gracePeriod=25
Nov 04 02:21:58.905826 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:58.905789    4969 generic.go:356] "Generic (PLEG): container finished" podID="6fc4dede-0532-4749-8cf2-44d868cd64c0" containerID="5f69836c0421bf2b5d89d47c02103fdb43828ebdcd749a992b777d527ec913ee" exitCode=0
Nov 04 02:21:58.906014 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:58.905895    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl" event={"ID":"6fc4dede-0532-4749-8cf2-44d868cd64c0","Type":"ContainerDied","Data":"5f69836c0421bf2b5d89d47c02103fdb43828ebdcd749a992b777d527ec913ee"}
Nov 04 02:21:58.991050 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:58.991014    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl"
Nov 04 02:21:59.079697 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:59.079663    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6fc4dede-0532-4749-8cf2-44d868cd64c0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fc4dede-0532-4749-8cf2-44d868cd64c0-plugins-config-volume\") pod \"6fc4dede-0532-4749-8cf2-44d868cd64c0\" (UID: \"6fc4dede-0532-4749-8cf2-44d868cd64c0\") "
Nov 04 02:21:59.079697 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:59.079708    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6fc4dede-0532-4749-8cf2-44d868cd64c0-kube-api-access-s69bl\" (UniqueName: \"kubernetes.io/projected/6fc4dede-0532-4749-8cf2-44d868cd64c0-kube-api-access-s69bl\") pod \"6fc4dede-0532-4749-8cf2-44d868cd64c0\" (UID: \"6fc4dede-0532-4749-8cf2-44d868cd64c0\") "
Nov 04 02:21:59.080405 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:59.080358    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fc4dede-0532-4749-8cf2-44d868cd64c0-plugins-config-volume" pod "6fc4dede-0532-4749-8cf2-44d868cd64c0" (UID: "6fc4dede-0532-4749-8cf2-44d868cd64c0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:21:59.082237 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:59.082133    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fc4dede-0532-4749-8cf2-44d868cd64c0-kube-api-access-s69bl" pod "6fc4dede-0532-4749-8cf2-44d868cd64c0" (UID: "6fc4dede-0532-4749-8cf2-44d868cd64c0"). InnerVolumeSpecName "kube-api-access-s69bl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:59.104185 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:59.104164    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc"
Nov 04 02:21:59.180743 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:59.180715    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/85d5905f-412e-4e4b-8553-9adb4a72afea-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85d5905f-412e-4e4b-8553-9adb4a72afea-plugins-config-volume\") pod \"85d5905f-412e-4e4b-8553-9adb4a72afea\" (UID: \"85d5905f-412e-4e4b-8553-9adb4a72afea\") "
Nov 04 02:21:59.180878 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:59.180809    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85d5905f-412e-4e4b-8553-9adb4a72afea-kube-api-access-vq45v\" (UniqueName: \"kubernetes.io/projected/85d5905f-412e-4e4b-8553-9adb4a72afea-kube-api-access-vq45v\") pod \"85d5905f-412e-4e4b-8553-9adb4a72afea\" (UID: \"85d5905f-412e-4e4b-8553-9adb4a72afea\") "
Nov 04 02:21:59.181069 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:59.181048    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85d5905f-412e-4e4b-8553-9adb4a72afea-plugins-config-volume" pod "85d5905f-412e-4e4b-8553-9adb4a72afea" (UID: "85d5905f-412e-4e4b-8553-9adb4a72afea"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:21:59.181163 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:59.181145    4969 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fc4dede-0532-4749-8cf2-44d868cd64c0-plugins-config-volume\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:59.181204 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:59.181171    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s69bl\" (UniqueName: \"kubernetes.io/projected/6fc4dede-0532-4749-8cf2-44d868cd64c0-kube-api-access-s69bl\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:59.182828 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:59.182811    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85d5905f-412e-4e4b-8553-9adb4a72afea-kube-api-access-vq45v" pod "85d5905f-412e-4e4b-8553-9adb4a72afea" (UID: "85d5905f-412e-4e4b-8553-9adb4a72afea"). InnerVolumeSpecName "kube-api-access-vq45v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:21:59.281866 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:59.281835    4969 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85d5905f-412e-4e4b-8553-9adb4a72afea-plugins-config-volume\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:59.281866 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:59.281864    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vq45v\" (UniqueName: \"kubernetes.io/projected/85d5905f-412e-4e4b-8553-9adb4a72afea-kube-api-access-vq45v\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:21:59.330023 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:59.329994    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:59.510673 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:59.510645    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:59.714390 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:59.714314    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:21:59.913764 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:59.913735    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl"
Nov 04 02:21:59.913910 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:59.913772    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl" event={"ID":"6fc4dede-0532-4749-8cf2-44d868cd64c0","Type":"ContainerDied","Data":"e23d79e5fec9b3428523c7de09048a60226727077ded91b11e46c5836ef26321"}
Nov 04 02:21:59.913910 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:59.913826    4969 scope.go:122] "RemoveContainer" containerID="5f69836c0421bf2b5d89d47c02103fdb43828ebdcd749a992b777d527ec913ee"
Nov 04 02:21:59.915287 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:59.915263    4969 generic.go:356] "Generic (PLEG): container finished" podID="85d5905f-412e-4e4b-8553-9adb4a72afea" containerID="450fa8d893d7e7bb5ec085d8b6b11d05e4c2e9bbf7ecb6eea2891070da55e042" exitCode=0
Nov 04 02:21:59.915401 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:59.915385    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc" event={"ID":"85d5905f-412e-4e4b-8553-9adb4a72afea","Type":"ContainerDied","Data":"450fa8d893d7e7bb5ec085d8b6b11d05e4c2e9bbf7ecb6eea2891070da55e042"}
Nov 04 02:21:59.915444 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:59.915432    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc" event={"ID":"85d5905f-412e-4e4b-8553-9adb4a72afea","Type":"ContainerDied","Data":"42e27fdb27378543e398018e821148eec01679b00100046f392aebc77b5a0357"}
Nov 04 02:21:59.915514 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:59.915500    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc"
Nov 04 02:21:59.925229 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:59.925207    4969 scope.go:122] "RemoveContainer" containerID="450fa8d893d7e7bb5ec085d8b6b11d05e4c2e9bbf7ecb6eea2891070da55e042"
Nov 04 02:21:59.933694 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:59.933676    4969 scope.go:122] "RemoveContainer" containerID="450fa8d893d7e7bb5ec085d8b6b11d05e4c2e9bbf7ecb6eea2891070da55e042"
Nov 04 02:21:59.933945 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:21:59.933925    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"450fa8d893d7e7bb5ec085d8b6b11d05e4c2e9bbf7ecb6eea2891070da55e042\": container with ID starting with 450fa8d893d7e7bb5ec085d8b6b11d05e4c2e9bbf7ecb6eea2891070da55e042 not found: ID does not exist" containerID="450fa8d893d7e7bb5ec085d8b6b11d05e4c2e9bbf7ecb6eea2891070da55e042"
Nov 04 02:21:59.933994 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:21:59.933953    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"450fa8d893d7e7bb5ec085d8b6b11d05e4c2e9bbf7ecb6eea2891070da55e042"} err="failed to get container status \"450fa8d893d7e7bb5ec085d8b6b11d05e4c2e9bbf7ecb6eea2891070da55e042\": rpc error: code = NotFound desc = could not find container \"450fa8d893d7e7bb5ec085d8b6b11d05e4c2e9bbf7ecb6eea2891070da55e042\": container with ID starting with 450fa8d893d7e7bb5ec085d8b6b11d05e4c2e9bbf7ecb6eea2891070da55e042 not found: ID does not exist"
Nov 04 02:22:00.078627 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:00.078602    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc"]
Nov 04 02:22:00.128255 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:00.128229    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-5a27ba-58875f7f9f-k2ztc"]
Nov 04 02:22:00.193692 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:00.193663    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl"]
Nov 04 02:22:00.205997 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:00.205968    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="85d5905f-412e-4e4b-8553-9adb4a72afea" path="/var/lib/kubelet/pods/85d5905f-412e-4e4b-8553-9adb4a72afea/volumes"
Nov 04 02:22:00.206338 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:00.206324    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-5a27ba-5b57754bc6-l5mjl"]
Nov 04 02:22:00.526671 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:00.526643    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:01.411175 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:01.411144    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:02.202656 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:02.202622    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6fc4dede-0532-4749-8cf2-44d868cd64c0" path="/var/lib/kubelet/pods/6fc4dede-0532-4749-8cf2-44d868cd64c0/volumes"
Nov 04 02:22:04.261811 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:22:04.261774    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:22:04.737783 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:04.737706    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:04.952774 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:04.952742    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:05.130448 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:05.130413    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:06.830346 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:06.830308    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:07.018719 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:07.018667    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:07.239731 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:07.239642    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:07.613066 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:07.613026    4969 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc" podUID="756a3073-fe07-4efa-9469-3862ab9350d4" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:22:10.133422 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:10.133386    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:11.013854 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:11.013825    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:11.198821 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:11.198784    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:12.416999 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:12.416968    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:12.691520 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:12.691439    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:12.869790 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:12.869746    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:13.977914 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:13.977878    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:14.280858 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:14.280820    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:14.296704 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:14.296675    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh"]
Nov 04 02:22:14.296936 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:14.296902    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh" podUID="0858fde0-e137-4c9a-a19b-99f70732dcf5" containerName="maas-api" containerID="cri-o://584660ccaf82e36a5a324c4b8b905a22a6f9925c59f4301372245452515ca788" gracePeriod=30
Nov 04 02:22:14.463484 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:14.463453    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:14.596053 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:14.596025    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh"
Nov 04 02:22:14.760885 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:14.760841    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0858fde0-e137-4c9a-a19b-99f70732dcf5-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0858fde0-e137-4c9a-a19b-99f70732dcf5-metrics-certs\") pod \"0858fde0-e137-4c9a-a19b-99f70732dcf5\" (UID: \"0858fde0-e137-4c9a-a19b-99f70732dcf5\") "
Nov 04 02:22:14.761571 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:14.761547    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0858fde0-e137-4c9a-a19b-99f70732dcf5-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0858fde0-e137-4c9a-a19b-99f70732dcf5-maas-api-tls\") pod \"0858fde0-e137-4c9a-a19b-99f70732dcf5\" (UID: \"0858fde0-e137-4c9a-a19b-99f70732dcf5\") "
Nov 04 02:22:14.761828 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:14.761807    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0858fde0-e137-4c9a-a19b-99f70732dcf5-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0858fde0-e137-4c9a-a19b-99f70732dcf5-openshift-service-ca\") pod \"0858fde0-e137-4c9a-a19b-99f70732dcf5\" (UID: \"0858fde0-e137-4c9a-a19b-99f70732dcf5\") "
Nov 04 02:22:14.762180 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:14.762150    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0858fde0-e137-4c9a-a19b-99f70732dcf5-kube-api-access-qpjzm\" (UniqueName: \"kubernetes.io/projected/0858fde0-e137-4c9a-a19b-99f70732dcf5-kube-api-access-qpjzm\") pod \"0858fde0-e137-4c9a-a19b-99f70732dcf5\" (UID: \"0858fde0-e137-4c9a-a19b-99f70732dcf5\") "
Nov 04 02:22:14.763759 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:14.763720    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0858fde0-e137-4c9a-a19b-99f70732dcf5-openshift-service-ca" pod "0858fde0-e137-4c9a-a19b-99f70732dcf5" (UID: "0858fde0-e137-4c9a-a19b-99f70732dcf5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:22:14.788555 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:14.788529    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0858fde0-e137-4c9a-a19b-99f70732dcf5-maas-api-tls" pod "0858fde0-e137-4c9a-a19b-99f70732dcf5" (UID: "0858fde0-e137-4c9a-a19b-99f70732dcf5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:22:14.789367 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:14.789334    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0858fde0-e137-4c9a-a19b-99f70732dcf5-metrics-certs" pod "0858fde0-e137-4c9a-a19b-99f70732dcf5" (UID: "0858fde0-e137-4c9a-a19b-99f70732dcf5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:22:14.789672 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:14.789635    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0858fde0-e137-4c9a-a19b-99f70732dcf5-kube-api-access-qpjzm" pod "0858fde0-e137-4c9a-a19b-99f70732dcf5" (UID: "0858fde0-e137-4c9a-a19b-99f70732dcf5"). InnerVolumeSpecName "kube-api-access-qpjzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:22:14.864216 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:14.864083    4969 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0858fde0-e137-4c9a-a19b-99f70732dcf5-metrics-certs\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:14.864216 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:14.864164    4969 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0858fde0-e137-4c9a-a19b-99f70732dcf5-maas-api-tls\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:14.864216 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:14.864183    4969 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0858fde0-e137-4c9a-a19b-99f70732dcf5-openshift-service-ca\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:14.864216 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:14.864198    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpjzm\" (UniqueName: \"kubernetes.io/projected/0858fde0-e137-4c9a-a19b-99f70732dcf5-kube-api-access-qpjzm\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:15.020120 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:15.019800    4969 generic.go:356] "Generic (PLEG): container finished" podID="0858fde0-e137-4c9a-a19b-99f70732dcf5" containerID="584660ccaf82e36a5a324c4b8b905a22a6f9925c59f4301372245452515ca788" exitCode=0
Nov 04 02:22:15.020120 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:15.020014    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh" event={"ID":"0858fde0-e137-4c9a-a19b-99f70732dcf5","Type":"ContainerDied","Data":"584660ccaf82e36a5a324c4b8b905a22a6f9925c59f4301372245452515ca788"}
Nov 04 02:22:15.020120 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:15.020067    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh" event={"ID":"0858fde0-e137-4c9a-a19b-99f70732dcf5","Type":"ContainerDied","Data":"ddf24e3af5835934a6e529563a4554fef457a406c3e86c654b1bd73c19cb9ca8"}
Nov 04 02:22:15.025181 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:15.024686    4969 scope.go:122] "RemoveContainer" containerID="584660ccaf82e36a5a324c4b8b905a22a6f9925c59f4301372245452515ca788"
Nov 04 02:22:15.025181 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:15.024882    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh"
Nov 04 02:22:15.038185 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:15.036907    4969 scope.go:122] "RemoveContainer" containerID="584660ccaf82e36a5a324c4b8b905a22a6f9925c59f4301372245452515ca788"
Nov 04 02:22:15.038185 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:22:15.037304    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"584660ccaf82e36a5a324c4b8b905a22a6f9925c59f4301372245452515ca788\": container with ID starting with 584660ccaf82e36a5a324c4b8b905a22a6f9925c59f4301372245452515ca788 not found: ID does not exist" containerID="584660ccaf82e36a5a324c4b8b905a22a6f9925c59f4301372245452515ca788"
Nov 04 02:22:15.038185 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:15.037347    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"584660ccaf82e36a5a324c4b8b905a22a6f9925c59f4301372245452515ca788"} err="failed to get container status \"584660ccaf82e36a5a324c4b8b905a22a6f9925c59f4301372245452515ca788\": rpc error: code = NotFound desc = could not find container \"584660ccaf82e36a5a324c4b8b905a22a6f9925c59f4301372245452515ca788\": container with ID starting with 584660ccaf82e36a5a324c4b8b905a22a6f9925c59f4301372245452515ca788 not found: ID does not exist"
Nov 04 02:22:15.214823 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:15.214730    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh"]
Nov 04 02:22:15.263122 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:15.263054    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-9b0f0d-699b47bb4-2d8fh"]
Nov 04 02:22:15.639600 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:15.639551    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:16.202987 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:16.202945    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0858fde0-e137-4c9a-a19b-99f70732dcf5" path="/var/lib/kubelet/pods/0858fde0-e137-4c9a-a19b-99f70732dcf5/volumes"
Nov 04 02:22:16.455794 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:16.455660    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:16.536312 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:16.536285    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc"
Nov 04 02:22:16.706164 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:16.695689    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/756a3073-fe07-4efa-9469-3862ab9350d4-kube-api-access-qw2zv\" (UniqueName: \"kubernetes.io/projected/756a3073-fe07-4efa-9469-3862ab9350d4-kube-api-access-qw2zv\") pod \"756a3073-fe07-4efa-9469-3862ab9350d4\" (UID: \"756a3073-fe07-4efa-9469-3862ab9350d4\") "
Nov 04 02:22:16.706164 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:16.695818    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/756a3073-fe07-4efa-9469-3862ab9350d4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/756a3073-fe07-4efa-9469-3862ab9350d4-plugins-config-volume\") pod \"756a3073-fe07-4efa-9469-3862ab9350d4\" (UID: \"756a3073-fe07-4efa-9469-3862ab9350d4\") "
Nov 04 02:22:16.706413 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:16.706305    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/756a3073-fe07-4efa-9469-3862ab9350d4-plugins-config-volume" pod "756a3073-fe07-4efa-9469-3862ab9350d4" (UID: "756a3073-fe07-4efa-9469-3862ab9350d4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:22:16.799312 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:16.799272    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/756a3073-fe07-4efa-9469-3862ab9350d4-kube-api-access-qw2zv" pod "756a3073-fe07-4efa-9469-3862ab9350d4" (UID: "756a3073-fe07-4efa-9469-3862ab9350d4"). InnerVolumeSpecName "kube-api-access-qw2zv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:22:16.818435 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:16.799845    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qw2zv\" (UniqueName: \"kubernetes.io/projected/756a3073-fe07-4efa-9469-3862ab9350d4-kube-api-access-qw2zv\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:16.818435 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:16.799887    4969 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/756a3073-fe07-4efa-9469-3862ab9350d4-plugins-config-volume\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:17.037262 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:17.037217    4969 generic.go:356] "Generic (PLEG): container finished" podID="756a3073-fe07-4efa-9469-3862ab9350d4" containerID="557bc90183d721cc7f94097a91683ad14397aede4c52cd541be0b45f2307ae4a" exitCode=0
Nov 04 02:22:17.037436 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:17.037283    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc"
Nov 04 02:22:17.037436 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:17.037364    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc" event={"ID":"756a3073-fe07-4efa-9469-3862ab9350d4","Type":"ContainerDied","Data":"557bc90183d721cc7f94097a91683ad14397aede4c52cd541be0b45f2307ae4a"}
Nov 04 02:22:17.037567 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:17.037448    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc" event={"ID":"756a3073-fe07-4efa-9469-3862ab9350d4","Type":"ContainerDied","Data":"58d5a3a98b96e7531a19e4478eb4169da5ac941a9f7065e76b4e6b58d06b9d92"}
Nov 04 02:22:17.037567 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:17.037484    4969 scope.go:122] "RemoveContainer" containerID="557bc90183d721cc7f94097a91683ad14397aede4c52cd541be0b45f2307ae4a"
Nov 04 02:22:17.051691 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:17.051639    4969 scope.go:122] "RemoveContainer" containerID="557bc90183d721cc7f94097a91683ad14397aede4c52cd541be0b45f2307ae4a"
Nov 04 02:22:17.051962 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:22:17.051936    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"557bc90183d721cc7f94097a91683ad14397aede4c52cd541be0b45f2307ae4a\": container with ID starting with 557bc90183d721cc7f94097a91683ad14397aede4c52cd541be0b45f2307ae4a not found: ID does not exist" containerID="557bc90183d721cc7f94097a91683ad14397aede4c52cd541be0b45f2307ae4a"
Nov 04 02:22:17.052059 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:17.051972    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"557bc90183d721cc7f94097a91683ad14397aede4c52cd541be0b45f2307ae4a"} err="failed to get container status \"557bc90183d721cc7f94097a91683ad14397aede4c52cd541be0b45f2307ae4a\": rpc error: code = NotFound desc = could not find container \"557bc90183d721cc7f94097a91683ad14397aede4c52cd541be0b45f2307ae4a\": container with ID starting with 557bc90183d721cc7f94097a91683ad14397aede4c52cd541be0b45f2307ae4a not found: ID does not exist"
Nov 04 02:22:17.265933 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:17.265892    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc"]
Nov 04 02:22:17.316364 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:17.316284    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-8cf267-78bc48b99b-vtsfc"]
Nov 04 02:22:17.568758 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:17.568679    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:18.202978 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:18.202936    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="756a3073-fe07-4efa-9469-3862ab9350d4" path="/var/lib/kubelet/pods/756a3073-fe07-4efa-9469-3862ab9350d4/volumes"
Nov 04 02:22:19.474235 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:19.474198    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:19.669760 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:19.669730    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:19.846780 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:19.846748    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:20.636062 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:20.636029    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:21.035267 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:21.035226    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:21.573383 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:21.573354    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:21.885300 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:21.885223    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2"]
Nov 04 02:22:21.885679 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:21.885410    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2" podUID="50d5e35c-7ec2-4285-b10a-82e94e9f608f" containerName="maas-api" containerID="cri-o://2a56e0e67034e605b5629a2b72b7ae58b46d808627bc90f474947d63a66fddf1" gracePeriod=30
Nov 04 02:22:22.051344 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:22.051305    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:22.071201 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:22.071176    4969 generic.go:356] "Generic (PLEG): container finished" podID="50d5e35c-7ec2-4285-b10a-82e94e9f608f" containerID="2a56e0e67034e605b5629a2b72b7ae58b46d808627bc90f474947d63a66fddf1" exitCode=0
Nov 04 02:22:22.071329 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:22.071237    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2" event={"ID":"50d5e35c-7ec2-4285-b10a-82e94e9f608f","Type":"ContainerDied","Data":"2a56e0e67034e605b5629a2b72b7ae58b46d808627bc90f474947d63a66fddf1"}
Nov 04 02:22:22.144727 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:22.144672    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2"
Nov 04 02:22:22.166180 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:22.166151    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/50d5e35c-7ec2-4285-b10a-82e94e9f608f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50d5e35c-7ec2-4285-b10a-82e94e9f608f-openshift-service-ca\") pod \"50d5e35c-7ec2-4285-b10a-82e94e9f608f\" (UID: \"50d5e35c-7ec2-4285-b10a-82e94e9f608f\") "
Nov 04 02:22:22.166327 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:22.166200    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/50d5e35c-7ec2-4285-b10a-82e94e9f608f-kube-api-access-fcjc6\" (UniqueName: \"kubernetes.io/projected/50d5e35c-7ec2-4285-b10a-82e94e9f608f-kube-api-access-fcjc6\") pod \"50d5e35c-7ec2-4285-b10a-82e94e9f608f\" (UID: \"50d5e35c-7ec2-4285-b10a-82e94e9f608f\") "
Nov 04 02:22:22.166380 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:22.166323    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/50d5e35c-7ec2-4285-b10a-82e94e9f608f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/50d5e35c-7ec2-4285-b10a-82e94e9f608f-metrics-certs\") pod \"50d5e35c-7ec2-4285-b10a-82e94e9f608f\" (UID: \"50d5e35c-7ec2-4285-b10a-82e94e9f608f\") "
Nov 04 02:22:22.166451 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:22.166432    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/50d5e35c-7ec2-4285-b10a-82e94e9f608f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50d5e35c-7ec2-4285-b10a-82e94e9f608f-maas-api-tls\") pod \"50d5e35c-7ec2-4285-b10a-82e94e9f608f\" (UID: \"50d5e35c-7ec2-4285-b10a-82e94e9f608f\") "
Nov 04 02:22:22.166583 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:22.166558    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50d5e35c-7ec2-4285-b10a-82e94e9f608f-openshift-service-ca" pod "50d5e35c-7ec2-4285-b10a-82e94e9f608f" (UID: "50d5e35c-7ec2-4285-b10a-82e94e9f608f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:22:22.166961 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:22.166928    4969 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50d5e35c-7ec2-4285-b10a-82e94e9f608f-openshift-service-ca\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:22.172253 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:22.172229    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50d5e35c-7ec2-4285-b10a-82e94e9f608f-kube-api-access-fcjc6" pod "50d5e35c-7ec2-4285-b10a-82e94e9f608f" (UID: "50d5e35c-7ec2-4285-b10a-82e94e9f608f"). InnerVolumeSpecName "kube-api-access-fcjc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:22:22.172359 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:22.172252    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50d5e35c-7ec2-4285-b10a-82e94e9f608f-metrics-certs" pod "50d5e35c-7ec2-4285-b10a-82e94e9f608f" (UID: "50d5e35c-7ec2-4285-b10a-82e94e9f608f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:22:22.172359 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:22.172230    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50d5e35c-7ec2-4285-b10a-82e94e9f608f-maas-api-tls" pod "50d5e35c-7ec2-4285-b10a-82e94e9f608f" (UID: "50d5e35c-7ec2-4285-b10a-82e94e9f608f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:22:22.269168 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:22.269023    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcjc6\" (UniqueName: \"kubernetes.io/projected/50d5e35c-7ec2-4285-b10a-82e94e9f608f-kube-api-access-fcjc6\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:22.269168 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:22.269062    4969 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50d5e35c-7ec2-4285-b10a-82e94e9f608f-metrics-certs\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:22.269673 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:22.269644    4969 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50d5e35c-7ec2-4285-b10a-82e94e9f608f-maas-api-tls\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:22.962299 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:22.962256    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:23.077880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:23.077852    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2"
Nov 04 02:22:23.078050 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:23.077876    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2" event={"ID":"50d5e35c-7ec2-4285-b10a-82e94e9f608f","Type":"ContainerDied","Data":"2111d22f4bf8321c18a96f7f80323e6265e83e5f37dafe9eed7ea1da934a9cfa"}
Nov 04 02:22:23.078050 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:23.077967    4969 scope.go:122] "RemoveContainer" containerID="2a56e0e67034e605b5629a2b72b7ae58b46d808627bc90f474947d63a66fddf1"
Nov 04 02:22:23.245943 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:23.245866    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2"]
Nov 04 02:22:23.295483 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:23.295451    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-5a27ba-7759884ddd-xqcv2"]
Nov 04 02:22:23.360543 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:23.360505    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:24.205329 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:24.205287    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="50d5e35c-7ec2-4285-b10a-82e94e9f608f" path="/var/lib/kubelet/pods/50d5e35c-7ec2-4285-b10a-82e94e9f608f/volumes"
Nov 04 02:22:25.262088 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.262056    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:25.864714 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.864680    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:25.958858 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.958826    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"]
Nov 04 02:22:25.959602 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.959581    4969 cpu_manager.go:414] "containerMap: removing container" podUID="0858fde0-e137-4c9a-a19b-99f70732dcf5" containerName="maas-api"
Nov 04 02:22:25.959665 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.959604    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0858fde0-e137-4c9a-a19b-99f70732dcf5" containerName="maas-api"
Nov 04 02:22:25.959665 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.959631    4969 cpu_manager.go:414] "containerMap: removing container" podUID="f3dba90b-2c06-4bf8-8b89-525dcada1934" containerName="main"
Nov 04 02:22:25.959665 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.959639    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f3dba90b-2c06-4bf8-8b89-525dcada1934" containerName="main"
Nov 04 02:22:25.959665 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.959657    4969 cpu_manager.go:414] "containerMap: removing container" podUID="85d5905f-412e-4e4b-8553-9adb4a72afea" containerName="payload-pre-processing"
Nov 04 02:22:25.959797 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.959665    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85d5905f-412e-4e4b-8553-9adb4a72afea" containerName="payload-pre-processing"
Nov 04 02:22:25.959797 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.959678    4969 cpu_manager.go:414] "containerMap: removing container" podUID="6e246b7c-de7c-42be-b07c-0d95bd019886" containerName="main"
Nov 04 02:22:25.959797 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.959685    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6e246b7c-de7c-42be-b07c-0d95bd019886" containerName="main"
Nov 04 02:22:25.959797 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.959719    4969 cpu_manager.go:414] "containerMap: removing container" podUID="440b09dc-d2a6-4879-8b71-e189dd565c27" containerName="payload-processing"
Nov 04 02:22:25.959797 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.959726    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="440b09dc-d2a6-4879-8b71-e189dd565c27" containerName="payload-processing"
Nov 04 02:22:25.959797 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.959737    4969 cpu_manager.go:414] "containerMap: removing container" podUID="50d5e35c-7ec2-4285-b10a-82e94e9f608f" containerName="maas-api"
Nov 04 02:22:25.959797 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.959745    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="50d5e35c-7ec2-4285-b10a-82e94e9f608f" containerName="maas-api"
Nov 04 02:22:25.959797 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.959753    4969 cpu_manager.go:414] "containerMap: removing container" podUID="a3d654e3-9e45-46e3-bad8-cde2b543684f" containerName="main"
Nov 04 02:22:25.959797 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.959760    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a3d654e3-9e45-46e3-bad8-cde2b543684f" containerName="main"
Nov 04 02:22:25.959797 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.959770    4969 cpu_manager.go:414] "containerMap: removing container" podUID="4f1779bb-f57c-4235-9ef9-baf26eafbca7" containerName="main"
Nov 04 02:22:25.959797 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.959777    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4f1779bb-f57c-4235-9ef9-baf26eafbca7" containerName="main"
Nov 04 02:22:25.959797 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.959791    4969 cpu_manager.go:414] "containerMap: removing container" podUID="43d4ca4c-da17-4f49-898e-d08e931bc93b" containerName="payload-pre-processing"
Nov 04 02:22:25.959797 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.959798    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="43d4ca4c-da17-4f49-898e-d08e931bc93b" containerName="payload-pre-processing"
Nov 04 02:22:25.960198 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.959807    4969 cpu_manager.go:414] "containerMap: removing container" podUID="756a3073-fe07-4efa-9469-3862ab9350d4" containerName="payload-processing"
Nov 04 02:22:25.960198 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.959814    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="756a3073-fe07-4efa-9469-3862ab9350d4" containerName="payload-processing"
Nov 04 02:22:25.960198 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.959825    4969 cpu_manager.go:414] "containerMap: removing container" podUID="6fc4dede-0532-4749-8cf2-44d868cd64c0" containerName="payload-processing"
Nov 04 02:22:25.960198 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.959832    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6fc4dede-0532-4749-8cf2-44d868cd64c0" containerName="payload-processing"
Nov 04 02:22:25.960198 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.959952    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="85d5905f-412e-4e4b-8553-9adb4a72afea" containerName="payload-pre-processing"
Nov 04 02:22:25.960198 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.959964    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="6e246b7c-de7c-42be-b07c-0d95bd019886" containerName="main"
Nov 04 02:22:25.960198 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.959976    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="43d4ca4c-da17-4f49-898e-d08e931bc93b" containerName="payload-pre-processing"
Nov 04 02:22:25.960198 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.959985    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="440b09dc-d2a6-4879-8b71-e189dd565c27" containerName="payload-processing"
Nov 04 02:22:25.960198 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.959996    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="50d5e35c-7ec2-4285-b10a-82e94e9f608f" containerName="maas-api"
Nov 04 02:22:25.960198 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.960004    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="a3d654e3-9e45-46e3-bad8-cde2b543684f" containerName="main"
Nov 04 02:22:25.960198 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.960013    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="4f1779bb-f57c-4235-9ef9-baf26eafbca7" containerName="main"
Nov 04 02:22:25.960198 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.960023    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="756a3073-fe07-4efa-9469-3862ab9350d4" containerName="payload-processing"
Nov 04 02:22:25.960198 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.960032    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="6fc4dede-0532-4749-8cf2-44d868cd64c0" containerName="payload-processing"
Nov 04 02:22:25.960198 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.960045    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="0858fde0-e137-4c9a-a19b-99f70732dcf5" containerName="maas-api"
Nov 04 02:22:25.960198 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.960057    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="f3dba90b-2c06-4bf8-8b89-525dcada1934" containerName="main"
Nov 04 02:22:25.965298 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:25.965280    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.006931 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.006907    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-workload-certs\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.007062 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.006943    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9bt7\" (UniqueName: \"kubernetes.io/projected/189a997c-d6a8-4bd6-b048-b8f301544a0e-kube-api-access-q9bt7\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.007121 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.007079    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-istio-envoy\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.007192 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.007177    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-workload-socket\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.007239 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.007209    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/189a997c-d6a8-4bd6-b048-b8f301544a0e-istiod-ca-cert\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.007293 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.007277    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/189a997c-d6a8-4bd6-b048-b8f301544a0e-istio-podinfo\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.007419 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.007401    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-credential-socket\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.007479 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.007432    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-istio-data\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.007479 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.007458    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/189a997c-d6a8-4bd6-b048-b8f301544a0e-istio-token\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.041718 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.041693    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:26.066627 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.066603    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-8cf267-openshift-default-dockercfg-7jsf4\""
Nov 04 02:22:26.071252 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.071226    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"]
Nov 04 02:22:26.109036 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.109011    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/189a997c-d6a8-4bd6-b048-b8f301544a0e-istiod-ca-cert\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.109191 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.109043    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/189a997c-d6a8-4bd6-b048-b8f301544a0e-istio-podinfo\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.109191 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.109081    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-credential-socket\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.109191 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.109134    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-istio-data\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.109191 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.109165    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/189a997c-d6a8-4bd6-b048-b8f301544a0e-istio-token\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.109391 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.109273    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-workload-certs\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.109391 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.109305    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9bt7\" (UniqueName: \"kubernetes.io/projected/189a997c-d6a8-4bd6-b048-b8f301544a0e-kube-api-access-q9bt7\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.109391 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.109379    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-istio-envoy\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.109546 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.109430    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-workload-socket\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.109612 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.109544    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-istio-data\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.109747 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.109727    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-workload-socket\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.109821 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.109799    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-workload-certs\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.109944 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.109918    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-credential-socket\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.110043 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.109924    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/189a997c-d6a8-4bd6-b048-b8f301544a0e-istiod-ca-cert\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.111818 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.111778    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/189a997c-d6a8-4bd6-b048-b8f301544a0e-istio-podinfo\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.111903 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.111846    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-istio-envoy\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.170735 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.170668    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/189a997c-d6a8-4bd6-b048-b8f301544a0e-istio-token\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.170846 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.170827    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9bt7\" (UniqueName: \"kubernetes.io/projected/189a997c-d6a8-4bd6-b048-b8f301544a0e-kube-api-access-q9bt7\") pod \"e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") " pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.280292 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.280261    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:26.508508 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.508481    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"]
Nov 04 02:22:26.514372 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.514352    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"
Nov 04 02:22:26.580378 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.580347    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-dd5d18\"/\"openshift-service-ca.crt\""
Nov 04 02:22:26.580378 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.580368    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-dd5d18\"/\"kube-root-ca.crt\""
Nov 04 02:22:26.580607 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.580424    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-dd5d18\"/\"default-dockercfg-gszng\""
Nov 04 02:22:26.580607 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.580470    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-dd5d18\"/\"distinct-2-w5-dd5d18-kserve-self-signed-certs\""
Nov 04 02:22:26.586713 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.586683    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"]
Nov 04 02:22:26.614960 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.614934    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfcfd03f-e20c-47ef-b048-b6203c30f52c-tmp-dir\") pod \"distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg\" (UID: \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"
Nov 04 02:22:26.615137 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.614985    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfcfd03f-e20c-47ef-b048-b6203c30f52c-home\") pod \"distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg\" (UID: \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"
Nov 04 02:22:26.615215 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.615187    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfcfd03f-e20c-47ef-b048-b6203c30f52c-tls-certs\") pod \"distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg\" (UID: \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"
Nov 04 02:22:26.615257 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.615227    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfcfd03f-e20c-47ef-b048-b6203c30f52c-dshm\") pod \"distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg\" (UID: \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"
Nov 04 02:22:26.615354 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.615336    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfcfd03f-e20c-47ef-b048-b6203c30f52c-model-cache\") pod \"distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg\" (UID: \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"
Nov 04 02:22:26.615389 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.615373    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bb8t\" (UniqueName: \"kubernetes.io/projected/cfcfd03f-e20c-47ef-b048-b6203c30f52c-kube-api-access-9bb8t\") pod \"distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg\" (UID: \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"
Nov 04 02:22:26.631274 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.631251    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:26.714720 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.714695    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"]
Nov 04 02:22:26.716469 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.716447    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfcfd03f-e20c-47ef-b048-b6203c30f52c-home\") pod \"distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg\" (UID: \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"
Nov 04 02:22:26.716539 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.716525    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfcfd03f-e20c-47ef-b048-b6203c30f52c-tls-certs\") pod \"distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg\" (UID: \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"
Nov 04 02:22:26.716589 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.716551    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfcfd03f-e20c-47ef-b048-b6203c30f52c-dshm\") pod \"distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg\" (UID: \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"
Nov 04 02:22:26.716625 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.716588    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfcfd03f-e20c-47ef-b048-b6203c30f52c-model-cache\") pod \"distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg\" (UID: \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"
Nov 04 02:22:26.716625 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.716613    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bb8t\" (UniqueName: \"kubernetes.io/projected/cfcfd03f-e20c-47ef-b048-b6203c30f52c-kube-api-access-9bb8t\") pod \"distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg\" (UID: \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"
Nov 04 02:22:26.716722 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.716681    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfcfd03f-e20c-47ef-b048-b6203c30f52c-tmp-dir\") pod \"distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg\" (UID: \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"
Nov 04 02:22:26.717142 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.716901    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfcfd03f-e20c-47ef-b048-b6203c30f52c-home\") pod \"distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg\" (UID: \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"
Nov 04 02:22:26.717142 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.716977    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfcfd03f-e20c-47ef-b048-b6203c30f52c-tmp-dir\") pod \"distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg\" (UID: \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"
Nov 04 02:22:26.717142 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.717036    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfcfd03f-e20c-47ef-b048-b6203c30f52c-model-cache\") pod \"distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg\" (UID: \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"
Nov 04 02:22:26.717350 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:22:26.717153    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod189a997c_d6a8_4bd6_b048_b8f301544a0e.slice/crio-74c0930978ce5ae015a4d2d9bf82b815c9b492e9c6ca7d2658910ad6f25492a6 WatchSource:0}: Error finding container 74c0930978ce5ae015a4d2d9bf82b815c9b492e9c6ca7d2658910ad6f25492a6: Status 404 returned error can't find the container with id 74c0930978ce5ae015a4d2d9bf82b815c9b492e9c6ca7d2658910ad6f25492a6
Nov 04 02:22:26.718996 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.718976    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfcfd03f-e20c-47ef-b048-b6203c30f52c-dshm\") pod \"distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg\" (UID: \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"
Nov 04 02:22:26.719268 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.719249    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfcfd03f-e20c-47ef-b048-b6203c30f52c-tls-certs\") pod \"distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg\" (UID: \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"
Nov 04 02:22:26.719678 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.719650    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:22:26.719748 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.719734    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:22:26.719787 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.719768    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:22:26.779698 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.779666    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bb8t\" (UniqueName: \"kubernetes.io/projected/cfcfd03f-e20c-47ef-b048-b6203c30f52c-kube-api-access-9bb8t\") pod \"distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg\" (UID: \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\") " pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"
Nov 04 02:22:26.826759 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:26.826721    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"
Nov 04 02:22:27.110469 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:27.110389    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d" event={"ID":"189a997c-d6a8-4bd6-b048-b8f301544a0e","Type":"ContainerStarted","Data":"871e2cce79735e3ccd2f547051ab892a9c226501a62f0478e4dd864e396541ac"}
Nov 04 02:22:27.110613 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:27.110482    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d" event={"ID":"189a997c-d6a8-4bd6-b048-b8f301544a0e","Type":"ContainerStarted","Data":"74c0930978ce5ae015a4d2d9bf82b815c9b492e9c6ca7d2658910ad6f25492a6"}
Nov 04 02:22:27.224306 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:27.224277    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"]
Nov 04 02:22:27.225702 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:22:27.225670    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcfcfd03f_e20c_47ef_b048_b6203c30f52c.slice/crio-7ee28e8edfd5b9859b7c325494c6c331d3872fa621464f446ee355cd25b3d0cc WatchSource:0}: Error finding container 7ee28e8edfd5b9859b7c325494c6c331d3872fa621464f446ee355cd25b3d0cc: Status 404 returned error can't find the container with id 7ee28e8edfd5b9859b7c325494c6c331d3872fa621464f446ee355cd25b3d0cc
Nov 04 02:22:27.229951 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:27.229898    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d" podStartSLOduration=2.229879814 podStartE2EDuration="2.229879814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:22:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:22:27.22894725 +0000 UTC m=+983.155169895" watchObservedRunningTime="2026-11-04 02:22:27.229879814 +0000 UTC m=+983.156102448"
Nov 04 02:22:27.281232 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:27.281204    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:27.284051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:27.283397    4969 patch_prober.go:29] interesting pod/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 04 02:22:27.284051 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:27.283450    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d" podUID="189a997c-d6a8-4bd6-b048-b8f301544a0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 04 02:22:27.495758 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:27.495668    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:27.913961 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:27.913927    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"]
Nov 04 02:22:27.998117 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:27.997663    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:28.118445 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:28.118410    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg" event={"ID":"cfcfd03f-e20c-47ef-b048-b6203c30f52c","Type":"ContainerStarted","Data":"a1cb768c5792236200e47ae188f9c5b4a7a4bc8c8924a8f23438b77659f635f6"}
Nov 04 02:22:28.118445 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:28.118451    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg" event={"ID":"cfcfd03f-e20c-47ef-b048-b6203c30f52c","Type":"ContainerStarted","Data":"7ee28e8edfd5b9859b7c325494c6c331d3872fa621464f446ee355cd25b3d0cc"}
Nov 04 02:22:28.118834 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:28.118815    4969 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d" secret="" err="secret \"e2e-worker-w1-8cf267-openshift-default-dockercfg-7jsf4\" not found"
Nov 04 02:22:28.197173 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:28.193796    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:28.238806 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:28.238749    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg" podStartSLOduration=2.238737543 podStartE2EDuration="2.238737543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:22:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:22:28.2374226 +0000 UTC m=+984.163645233" watchObservedRunningTime="2026-11-04 02:22:28.238737543 +0000 UTC m=+984.164960176"
Nov 04 02:22:28.283216 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:28.281474    4969 patch_prober.go:29] interesting pod/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 04 02:22:28.283216 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:28.281538    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d" podUID="189a997c-d6a8-4bd6-b048-b8f301544a0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 04 02:22:28.827550 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:28.827513    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"
Nov 04 02:22:29.123765 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:29.123660    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d" podUID="189a997c-d6a8-4bd6-b048-b8f301544a0e" containerName="istio-proxy" containerID="cri-o://871e2cce79735e3ccd2f547051ab892a9c226501a62f0478e4dd864e396541ac" gracePeriod=30
Nov 04 02:22:29.682721 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:29.682683    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:30.142534 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:30.142502    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"]
Nov 04 02:22:30.142722 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:30.142700    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2" podUID="f1c402e7-f796-4c4a-af73-b9f07a9c98cc" containerName="istio-proxy" containerID="cri-o://c1f6c2343d35d7654764112ad8652f83e338c1a7688cd6264edd92209a517f23" gracePeriod=30
Nov 04 02:22:30.182856 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:30.182817    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:30.699974 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:30.699938    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:31.447967 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:31.447937    4969 patch_prober.go:29] interesting pod/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 02:22:31.450134 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:31.447984    4969 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2" podUID="f1c402e7-f796-4c4a-af73-b9f07a9c98cc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 02:22:31.779114 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:31.778637    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:32.041807 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:32.041723    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:33.279040 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:33.279001    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:33.784794 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:33.784764    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:34.262521 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:22:34.262474    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:22:34.607224 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:34.607150    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:35.405395 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.405372    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:22:35.540426 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.540393    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-workload-certs\") pod \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") "
Nov 04 02:22:35.540614 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.540492    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istio-data\") pod \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") "
Nov 04 02:22:35.540614 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.540518    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-credential-socket\") pod \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") "
Nov 04 02:22:35.540614 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.540581    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istio-podinfo\") pod \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") "
Nov 04 02:22:35.540614 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.540611    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istio-envoy\") pod \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") "
Nov 04 02:22:35.540834 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.540653    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-kube-api-access-6sts8\" (UniqueName: \"kubernetes.io/projected/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-kube-api-access-6sts8\") pod \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") "
Nov 04 02:22:35.540834 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.540705    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istiod-ca-cert\") pod \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") "
Nov 04 02:22:35.540834 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.540721    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-workload-certs" pod "f1c402e7-f796-4c4a-af73-b9f07a9c98cc" (UID: "f1c402e7-f796-4c4a-af73-b9f07a9c98cc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:35.540834 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.540791    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-workload-socket\") pod \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") "
Nov 04 02:22:35.541039 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.540834    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istio-data" pod "f1c402e7-f796-4c4a-af73-b9f07a9c98cc" (UID: "f1c402e7-f796-4c4a-af73-b9f07a9c98cc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:35.541039 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.540841    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istio-token\" (UniqueName: \"kubernetes.io/projected/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istio-token\") pod \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\" (UID: \"f1c402e7-f796-4c4a-af73-b9f07a9c98cc\") "
Nov 04 02:22:35.541039 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.540908    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-credential-socket" pod "f1c402e7-f796-4c4a-af73-b9f07a9c98cc" (UID: "f1c402e7-f796-4c4a-af73-b9f07a9c98cc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:35.541216 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.541199    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istiod-ca-cert" pod "f1c402e7-f796-4c4a-af73-b9f07a9c98cc" (UID: "f1c402e7-f796-4c4a-af73-b9f07a9c98cc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:22:35.541421 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.541339    4969 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istiod-ca-cert\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:35.541421 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.541362    4969 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-workload-certs\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:35.541421 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.541375    4969 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istio-data\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:35.541421 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.541391    4969 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-credential-socket\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:35.541584 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.541417    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-workload-socket" pod "f1c402e7-f796-4c4a-af73-b9f07a9c98cc" (UID: "f1c402e7-f796-4c4a-af73-b9f07a9c98cc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:35.543810 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.543784    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istio-podinfo" pod "f1c402e7-f796-4c4a-af73-b9f07a9c98cc" (UID: "f1c402e7-f796-4c4a-af73-b9f07a9c98cc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:22:35.543810 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.543793    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istio-envoy" pod "f1c402e7-f796-4c4a-af73-b9f07a9c98cc" (UID: "f1c402e7-f796-4c4a-af73-b9f07a9c98cc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:35.543938 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.543789    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-kube-api-access-6sts8" pod "f1c402e7-f796-4c4a-af73-b9f07a9c98cc" (UID: "f1c402e7-f796-4c4a-af73-b9f07a9c98cc"). InnerVolumeSpecName "kube-api-access-6sts8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:22:35.543938 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.543816    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istio-token" pod "f1c402e7-f796-4c4a-af73-b9f07a9c98cc" (UID: "f1c402e7-f796-4c4a-af73-b9f07a9c98cc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:22:35.642291 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.642259    4969 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-workload-socket\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:35.642291 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.642286    4969 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istio-token\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:35.642291 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.642296    4969 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istio-podinfo\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:35.642707 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.642306    4969 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-istio-envoy\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:35.642707 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.642315    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6sts8\" (UniqueName: \"kubernetes.io/projected/f1c402e7-f796-4c4a-af73-b9f07a9c98cc-kube-api-access-6sts8\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:35.992604 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:35.992524    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:36.171287 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:36.171252    4969 generic.go:356] "Generic (PLEG): container finished" podID="f1c402e7-f796-4c4a-af73-b9f07a9c98cc" containerID="c1f6c2343d35d7654764112ad8652f83e338c1a7688cd6264edd92209a517f23" exitCode=0
Nov 04 02:22:36.171481 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:36.171308    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"
Nov 04 02:22:36.171481 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:36.171335    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2" event={"ID":"f1c402e7-f796-4c4a-af73-b9f07a9c98cc","Type":"ContainerDied","Data":"c1f6c2343d35d7654764112ad8652f83e338c1a7688cd6264edd92209a517f23"}
Nov 04 02:22:36.171481 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:36.171390    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2" event={"ID":"f1c402e7-f796-4c4a-af73-b9f07a9c98cc","Type":"ContainerDied","Data":"50c30f29f81fb77e93fe2ba6c2bdf2a481ca9a372c609a11b63159955050d796"}
Nov 04 02:22:36.171481 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:36.171416    4969 scope.go:122] "RemoveContainer" containerID="c1f6c2343d35d7654764112ad8652f83e338c1a7688cd6264edd92209a517f23"
Nov 04 02:22:36.182291 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:36.182273    4969 scope.go:122] "RemoveContainer" containerID="c1f6c2343d35d7654764112ad8652f83e338c1a7688cd6264edd92209a517f23"
Nov 04 02:22:36.182572 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:22:36.182552    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1f6c2343d35d7654764112ad8652f83e338c1a7688cd6264edd92209a517f23\": container with ID starting with c1f6c2343d35d7654764112ad8652f83e338c1a7688cd6264edd92209a517f23 not found: ID does not exist" containerID="c1f6c2343d35d7654764112ad8652f83e338c1a7688cd6264edd92209a517f23"
Nov 04 02:22:36.182643 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:36.182579    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1f6c2343d35d7654764112ad8652f83e338c1a7688cd6264edd92209a517f23"} err="failed to get container status \"c1f6c2343d35d7654764112ad8652f83e338c1a7688cd6264edd92209a517f23\": rpc error: code = NotFound desc = could not find container \"c1f6c2343d35d7654764112ad8652f83e338c1a7688cd6264edd92209a517f23\": container with ID starting with c1f6c2343d35d7654764112ad8652f83e338c1a7688cd6264edd92209a517f23 not found: ID does not exist"
Nov 04 02:22:36.335914 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:36.335883    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"]
Nov 04 02:22:36.385416 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:36.385382    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-5a27ba-openshift-default-74b449ccf5-mxzn2"]
Nov 04 02:22:36.826974 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:36.826942    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"
Nov 04 02:22:36.840015 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:36.839990    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"
Nov 04 02:22:37.189728 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:37.189642    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"
Nov 04 02:22:37.723133 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:37.723088    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"]
Nov 04 02:22:37.723425 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:37.723361    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm" podUID="162cba19-31aa-4cf1-80d1-96ef638c8ee7" containerName="istio-proxy" containerID="cri-o://a5d04015b9da1b3245491fdfbda7ae7fc34473a7bf6b814cdfccf66d2ff5bdb2" gracePeriod=30
Nov 04 02:22:38.202451 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:38.202422    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f1c402e7-f796-4c4a-af73-b9f07a9c98cc" path="/var/lib/kubelet/pods/f1c402e7-f796-4c4a-af73-b9f07a9c98cc/volumes"
Nov 04 02:22:38.900169 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:38.900144    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:39.077037 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.077000    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/189a997c-d6a8-4bd6-b048-b8f301544a0e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/189a997c-d6a8-4bd6-b048-b8f301544a0e-istio-podinfo\") pod \"189a997c-d6a8-4bd6-b048-b8f301544a0e\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") "
Nov 04 02:22:39.077242 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.077054    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-istio-envoy\") pod \"189a997c-d6a8-4bd6-b048-b8f301544a0e\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") "
Nov 04 02:22:39.077242 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.077122    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-workload-certs\") pod \"189a997c-d6a8-4bd6-b048-b8f301544a0e\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") "
Nov 04 02:22:39.077242 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.077147    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/189a997c-d6a8-4bd6-b048-b8f301544a0e-kube-api-access-q9bt7\" (UniqueName: \"kubernetes.io/projected/189a997c-d6a8-4bd6-b048-b8f301544a0e-kube-api-access-q9bt7\") pod \"189a997c-d6a8-4bd6-b048-b8f301544a0e\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") "
Nov 04 02:22:39.077242 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.077177    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-workload-socket\") pod \"189a997c-d6a8-4bd6-b048-b8f301544a0e\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") "
Nov 04 02:22:39.077242 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.077222    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/189a997c-d6a8-4bd6-b048-b8f301544a0e-istio-token\" (UniqueName: \"kubernetes.io/projected/189a997c-d6a8-4bd6-b048-b8f301544a0e-istio-token\") pod \"189a997c-d6a8-4bd6-b048-b8f301544a0e\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") "
Nov 04 02:22:39.077537 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.077298    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/189a997c-d6a8-4bd6-b048-b8f301544a0e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/189a997c-d6a8-4bd6-b048-b8f301544a0e-istiod-ca-cert\") pod \"189a997c-d6a8-4bd6-b048-b8f301544a0e\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") "
Nov 04 02:22:39.077537 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.077327    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-credential-socket\") pod \"189a997c-d6a8-4bd6-b048-b8f301544a0e\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") "
Nov 04 02:22:39.077537 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.077408    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-istio-data\") pod \"189a997c-d6a8-4bd6-b048-b8f301544a0e\" (UID: \"189a997c-d6a8-4bd6-b048-b8f301544a0e\") "
Nov 04 02:22:39.077537 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.077442    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-workload-certs" pod "189a997c-d6a8-4bd6-b048-b8f301544a0e" (UID: "189a997c-d6a8-4bd6-b048-b8f301544a0e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:39.077778 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.077759    4969 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-workload-certs\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:39.077971 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.077940    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-workload-socket" pod "189a997c-d6a8-4bd6-b048-b8f301544a0e" (UID: "189a997c-d6a8-4bd6-b048-b8f301544a0e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:39.078056 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.078034    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-credential-socket" pod "189a997c-d6a8-4bd6-b048-b8f301544a0e" (UID: "189a997c-d6a8-4bd6-b048-b8f301544a0e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:39.078312 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.078289    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-istio-data" pod "189a997c-d6a8-4bd6-b048-b8f301544a0e" (UID: "189a997c-d6a8-4bd6-b048-b8f301544a0e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:39.078439 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.078412    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/189a997c-d6a8-4bd6-b048-b8f301544a0e-istiod-ca-cert" pod "189a997c-d6a8-4bd6-b048-b8f301544a0e" (UID: "189a997c-d6a8-4bd6-b048-b8f301544a0e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:22:39.079623 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.079602    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/189a997c-d6a8-4bd6-b048-b8f301544a0e-istio-podinfo" pod "189a997c-d6a8-4bd6-b048-b8f301544a0e" (UID: "189a997c-d6a8-4bd6-b048-b8f301544a0e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:22:39.080143 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.080122    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-istio-envoy" pod "189a997c-d6a8-4bd6-b048-b8f301544a0e" (UID: "189a997c-d6a8-4bd6-b048-b8f301544a0e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:39.080242 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.080168    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/189a997c-d6a8-4bd6-b048-b8f301544a0e-istio-token" pod "189a997c-d6a8-4bd6-b048-b8f301544a0e" (UID: "189a997c-d6a8-4bd6-b048-b8f301544a0e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:22:39.080533 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.080243    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/189a997c-d6a8-4bd6-b048-b8f301544a0e-kube-api-access-q9bt7" pod "189a997c-d6a8-4bd6-b048-b8f301544a0e" (UID: "189a997c-d6a8-4bd6-b048-b8f301544a0e"). InnerVolumeSpecName "kube-api-access-q9bt7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:22:39.179469 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.179435    4969 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/189a997c-d6a8-4bd6-b048-b8f301544a0e-istio-podinfo\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:39.179615 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.179471    4969 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-istio-envoy\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:39.179615 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.179488    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9bt7\" (UniqueName: \"kubernetes.io/projected/189a997c-d6a8-4bd6-b048-b8f301544a0e-kube-api-access-q9bt7\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:39.179615 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.179504    4969 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-workload-socket\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:39.179615 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.179520    4969 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/189a997c-d6a8-4bd6-b048-b8f301544a0e-istio-token\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:39.179615 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.179534    4969 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/189a997c-d6a8-4bd6-b048-b8f301544a0e-istiod-ca-cert\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:39.179615 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.179546    4969 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-credential-socket\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:39.179615 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.179567    4969 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/189a997c-d6a8-4bd6-b048-b8f301544a0e-istio-data\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:39.191769 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.191743    4969 generic.go:356] "Generic (PLEG): container finished" podID="189a997c-d6a8-4bd6-b048-b8f301544a0e" containerID="871e2cce79735e3ccd2f547051ab892a9c226501a62f0478e4dd864e396541ac" exitCode=0
Nov 04 02:22:39.191903 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.191790    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d" event={"ID":"189a997c-d6a8-4bd6-b048-b8f301544a0e","Type":"ContainerDied","Data":"871e2cce79735e3ccd2f547051ab892a9c226501a62f0478e4dd864e396541ac"}
Nov 04 02:22:39.191903 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.191837    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d" event={"ID":"189a997c-d6a8-4bd6-b048-b8f301544a0e","Type":"ContainerDied","Data":"74c0930978ce5ae015a4d2d9bf82b815c9b492e9c6ca7d2658910ad6f25492a6"}
Nov 04 02:22:39.191903 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.191868    4969 scope.go:122] "RemoveContainer" containerID="871e2cce79735e3ccd2f547051ab892a9c226501a62f0478e4dd864e396541ac"
Nov 04 02:22:39.192070 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.192000    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"
Nov 04 02:22:39.204145 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.204124    4969 scope.go:122] "RemoveContainer" containerID="871e2cce79735e3ccd2f547051ab892a9c226501a62f0478e4dd864e396541ac"
Nov 04 02:22:39.204474 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:22:39.204452    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"871e2cce79735e3ccd2f547051ab892a9c226501a62f0478e4dd864e396541ac\": container with ID starting with 871e2cce79735e3ccd2f547051ab892a9c226501a62f0478e4dd864e396541ac not found: ID does not exist" containerID="871e2cce79735e3ccd2f547051ab892a9c226501a62f0478e4dd864e396541ac"
Nov 04 02:22:39.204529 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.204487    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"871e2cce79735e3ccd2f547051ab892a9c226501a62f0478e4dd864e396541ac"} err="failed to get container status \"871e2cce79735e3ccd2f547051ab892a9c226501a62f0478e4dd864e396541ac\": rpc error: code = NotFound desc = could not find container \"871e2cce79735e3ccd2f547051ab892a9c226501a62f0478e4dd864e396541ac\": container with ID starting with 871e2cce79735e3ccd2f547051ab892a9c226501a62f0478e4dd864e396541ac not found: ID does not exist"
Nov 04 02:22:39.308182 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.308148    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:39.425654 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.425579    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"]
Nov 04 02:22:39.455318 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.455292    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-8cf267-openshift-default-7cb975b44c-q2p2d"]
Nov 04 02:22:39.629253 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:39.629222    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:40.202067 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:40.202028    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="189a997c-d6a8-4bd6-b048-b8f301544a0e" path="/var/lib/kubelet/pods/189a997c-d6a8-4bd6-b048-b8f301544a0e/volumes"
Nov 04 02:22:41.146414 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:41.146383    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:42.991608 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:42.991535    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:22:43.019416 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.019385    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istio-data\") pod \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") "
Nov 04 02:22:43.019539 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.019434    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/162cba19-31aa-4cf1-80d1-96ef638c8ee7-kube-api-access-fzw94\" (UniqueName: \"kubernetes.io/projected/162cba19-31aa-4cf1-80d1-96ef638c8ee7-kube-api-access-fzw94\") pod \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") "
Nov 04 02:22:43.019539 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.019471    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istio-token\" (UniqueName: \"kubernetes.io/projected/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istio-token\") pod \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") "
Nov 04 02:22:43.019539 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.019524    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istio-envoy\") pod \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") "
Nov 04 02:22:43.019689 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.019602    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-credential-socket\") pod \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") "
Nov 04 02:22:43.019689 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.019644    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-workload-certs\") pod \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") "
Nov 04 02:22:43.019798 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.019685    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istio-data" pod "162cba19-31aa-4cf1-80d1-96ef638c8ee7" (UID: "162cba19-31aa-4cf1-80d1-96ef638c8ee7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:43.019798 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.019690    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-workload-socket\") pod \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") "
Nov 04 02:22:43.019798 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.019742    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istiod-ca-cert\") pod \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") "
Nov 04 02:22:43.019954 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.019812    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istio-podinfo\") pod \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\" (UID: \"162cba19-31aa-4cf1-80d1-96ef638c8ee7\") "
Nov 04 02:22:43.019954 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.019929    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-workload-socket" pod "162cba19-31aa-4cf1-80d1-96ef638c8ee7" (UID: "162cba19-31aa-4cf1-80d1-96ef638c8ee7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:43.020072 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.020004    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-credential-socket" pod "162cba19-31aa-4cf1-80d1-96ef638c8ee7" (UID: "162cba19-31aa-4cf1-80d1-96ef638c8ee7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:43.020177 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.020115    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istiod-ca-cert" pod "162cba19-31aa-4cf1-80d1-96ef638c8ee7" (UID: "162cba19-31aa-4cf1-80d1-96ef638c8ee7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:22:43.020298 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.020277    4969 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istio-data\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:43.020359 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.020305    4969 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-credential-socket\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:43.020359 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.020321    4969 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-workload-socket\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:43.020359 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.020335    4969 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istiod-ca-cert\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:43.020527 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.020415    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-workload-certs" pod "162cba19-31aa-4cf1-80d1-96ef638c8ee7" (UID: "162cba19-31aa-4cf1-80d1-96ef638c8ee7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:43.021743 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.021716    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istio-envoy" pod "162cba19-31aa-4cf1-80d1-96ef638c8ee7" (UID: "162cba19-31aa-4cf1-80d1-96ef638c8ee7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:22:43.021853 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.021757    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/162cba19-31aa-4cf1-80d1-96ef638c8ee7-kube-api-access-fzw94" pod "162cba19-31aa-4cf1-80d1-96ef638c8ee7" (UID: "162cba19-31aa-4cf1-80d1-96ef638c8ee7"). InnerVolumeSpecName "kube-api-access-fzw94". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:22:43.021853 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.021828    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istio-token" pod "162cba19-31aa-4cf1-80d1-96ef638c8ee7" (UID: "162cba19-31aa-4cf1-80d1-96ef638c8ee7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:22:43.021937 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.021867    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istio-podinfo" pod "162cba19-31aa-4cf1-80d1-96ef638c8ee7" (UID: "162cba19-31aa-4cf1-80d1-96ef638c8ee7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:22:43.121084 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.121054    4969 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istio-envoy\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:43.121084 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.121080    4969 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/162cba19-31aa-4cf1-80d1-96ef638c8ee7-workload-certs\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:43.121252 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.121113    4969 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istio-podinfo\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:43.121252 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.121122    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzw94\" (UniqueName: \"kubernetes.io/projected/162cba19-31aa-4cf1-80d1-96ef638c8ee7-kube-api-access-fzw94\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:43.121252 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.121133    4969 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/162cba19-31aa-4cf1-80d1-96ef638c8ee7-istio-token\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:22:43.223454 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.223423    4969 generic.go:356] "Generic (PLEG): container finished" podID="162cba19-31aa-4cf1-80d1-96ef638c8ee7" containerID="a5d04015b9da1b3245491fdfbda7ae7fc34473a7bf6b814cdfccf66d2ff5bdb2" exitCode=0
Nov 04 02:22:43.223628 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.223473    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"
Nov 04 02:22:43.223628 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.223504    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm" event={"ID":"162cba19-31aa-4cf1-80d1-96ef638c8ee7","Type":"ContainerDied","Data":"a5d04015b9da1b3245491fdfbda7ae7fc34473a7bf6b814cdfccf66d2ff5bdb2"}
Nov 04 02:22:43.223628 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.223572    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm" event={"ID":"162cba19-31aa-4cf1-80d1-96ef638c8ee7","Type":"ContainerDied","Data":"a2d8fdd5a828769e887ab3ad8d4861aa547cafe24091329596d30548c7eea9a1"}
Nov 04 02:22:43.223628 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.223607    4969 scope.go:122] "RemoveContainer" containerID="a5d04015b9da1b3245491fdfbda7ae7fc34473a7bf6b814cdfccf66d2ff5bdb2"
Nov 04 02:22:43.234180 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.234155    4969 scope.go:122] "RemoveContainer" containerID="a5d04015b9da1b3245491fdfbda7ae7fc34473a7bf6b814cdfccf66d2ff5bdb2"
Nov 04 02:22:43.234517 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:22:43.234493    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5d04015b9da1b3245491fdfbda7ae7fc34473a7bf6b814cdfccf66d2ff5bdb2\": container with ID starting with a5d04015b9da1b3245491fdfbda7ae7fc34473a7bf6b814cdfccf66d2ff5bdb2 not found: ID does not exist" containerID="a5d04015b9da1b3245491fdfbda7ae7fc34473a7bf6b814cdfccf66d2ff5bdb2"
Nov 04 02:22:43.234588 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.234527    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5d04015b9da1b3245491fdfbda7ae7fc34473a7bf6b814cdfccf66d2ff5bdb2"} err="failed to get container status \"a5d04015b9da1b3245491fdfbda7ae7fc34473a7bf6b814cdfccf66d2ff5bdb2\": rpc error: code = NotFound desc = could not find container \"a5d04015b9da1b3245491fdfbda7ae7fc34473a7bf6b814cdfccf66d2ff5bdb2\": container with ID starting with a5d04015b9da1b3245491fdfbda7ae7fc34473a7bf6b814cdfccf66d2ff5bdb2 not found: ID does not exist"
Nov 04 02:22:43.388582 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.388541    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"]
Nov 04 02:22:43.394106 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:43.394067    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6cc458d887-628mm"]
Nov 04 02:22:44.104204 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:44.104161    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:44.203595 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:44.203548    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="162cba19-31aa-4cf1-80d1-96ef638c8ee7" path="/var/lib/kubelet/pods/162cba19-31aa-4cf1-80d1-96ef638c8ee7/volumes"
Nov 04 02:22:44.290136 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:44.290079    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:46.117833 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:46.117796    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:46.364370 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:46.364338    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:46.640643 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:46.640605    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:47.902485 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:47.902456    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:48.084683 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:48.084646    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:49.918637 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:49.918605    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:51.029727 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:51.029696    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:55.048573 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:55.048536    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:55.930586 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:55.930551    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:22:59.699067 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:22:59.699033    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:00.734306 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:00.734273    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:01.254757 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:01.254725    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:04.260645 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:23:04.260611    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:23:05.615896 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:05.615858    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:07.520703 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:07.520664    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:07.685211 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:07.685175    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx"]
Nov 04 02:23:07.858292 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:07.858213    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt"]
Nov 04 02:23:08.089876 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.089846    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:08.202293 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.202224    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"]
Nov 04 02:23:08.202746 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.202732    4969 cpu_manager.go:414] "containerMap: removing container" podUID="f1c402e7-f796-4c4a-af73-b9f07a9c98cc" containerName="istio-proxy"
Nov 04 02:23:08.202795 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.202748    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f1c402e7-f796-4c4a-af73-b9f07a9c98cc" containerName="istio-proxy"
Nov 04 02:23:08.202795 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.202759    4969 cpu_manager.go:414] "containerMap: removing container" podUID="189a997c-d6a8-4bd6-b048-b8f301544a0e" containerName="istio-proxy"
Nov 04 02:23:08.202795 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.202764    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="189a997c-d6a8-4bd6-b048-b8f301544a0e" containerName="istio-proxy"
Nov 04 02:23:08.202795 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.202777    4969 cpu_manager.go:414] "containerMap: removing container" podUID="162cba19-31aa-4cf1-80d1-96ef638c8ee7" containerName="istio-proxy"
Nov 04 02:23:08.202795 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.202783    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="162cba19-31aa-4cf1-80d1-96ef638c8ee7" containerName="istio-proxy"
Nov 04 02:23:08.204604 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.203642    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="f1c402e7-f796-4c4a-af73-b9f07a9c98cc" containerName="istio-proxy"
Nov 04 02:23:08.204604 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.203688    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="189a997c-d6a8-4bd6-b048-b8f301544a0e" containerName="istio-proxy"
Nov 04 02:23:08.204604 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.203709    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="162cba19-31aa-4cf1-80d1-96ef638c8ee7" containerName="istio-proxy"
Nov 04 02:23:08.210227 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.210203    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:08.263061 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.263031    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-dd5d18-openshift-default-dockercfg-5qdvr\""
Nov 04 02:23:08.264954 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.264926    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:08.267013 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.266991    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"]
Nov 04 02:23:08.369161 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.369132    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-workload-certs\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:08.369321 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.369170    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e50fee5-a7b8-436b-8685-ff90f478d369-istio-podinfo\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:08.369321 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.369240    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-credential-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:08.369321 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.369310    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e50fee5-a7b8-436b-8685-ff90f478d369-istio-token\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:08.369426 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.369334    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvgc6\" (UniqueName: \"kubernetes.io/projected/4e50fee5-a7b8-436b-8685-ff90f478d369-kube-api-access-cvgc6\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:08.369426 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.369394    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-istio-data\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:08.369493 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.369456    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-workload-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:08.369493 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.369475    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-istio-envoy\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:08.369556 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.369493    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e50fee5-a7b8-436b-8685-ff90f478d369-istiod-ca-cert\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:08.470586 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.470510    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-workload-certs\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:08.470586 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.470548    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e50fee5-a7b8-436b-8685-ff90f478d369-istio-podinfo\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:08.470586 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.470570    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-credential-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:08.470797 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.470661    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e50fee5-a7b8-436b-8685-ff90f478d369-istio-token\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:08.470797 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.470685    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvgc6\" (UniqueName: \"kubernetes.io/projected/4e50fee5-a7b8-436b-8685-ff90f478d369-kube-api-access-cvgc6\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:08.470797 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.470720    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-istio-data\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:08.470797 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.470768    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-workload-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:08.471026 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.470945    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-workload-certs\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:08.471026 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.471016    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-istio-envoy\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:08.471247 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.470972    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-credential-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:08.471247 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.471061    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e50fee5-a7b8-436b-8685-ff90f478d369-istiod-ca-cert\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:08.471247 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.471159    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-workload-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:08.471247 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.471216    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-istio-data\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:08.471620 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.471603    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e50fee5-a7b8-436b-8685-ff90f478d369-istiod-ca-cert\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:08.473118 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.473080    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e50fee5-a7b8-436b-8685-ff90f478d369-istio-podinfo\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:08.473196 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.473132    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-istio-envoy\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:08.550234 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.550205    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e50fee5-a7b8-436b-8685-ff90f478d369-istio-token\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:08.550531 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.550419    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvgc6\" (UniqueName: \"kubernetes.io/projected/4e50fee5-a7b8-436b-8685-ff90f478d369-kube-api-access-cvgc6\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:08.824276 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:08.824245    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:09.261520 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:23:09.261486    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e50fee5_a7b8_436b_8685_ff90f478d369.slice/crio-92ec2fcfa8b6ad8b432df874eb09728524e691957d0a420b5b45f83d7ef92cf4 WatchSource:0}: Error finding container 92ec2fcfa8b6ad8b432df874eb09728524e691957d0a420b5b45f83d7ef92cf4: Status 404 returned error can't find the container with id 92ec2fcfa8b6ad8b432df874eb09728524e691957d0a420b5b45f83d7ef92cf4
Nov 04 02:23:09.263808 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:09.263773    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:23:09.263925 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:09.263858    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:23:09.263925 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:09.263903    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:23:09.304664 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:09.304641    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"]
Nov 04 02:23:09.379207 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:09.379170    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns" event={"ID":"4e50fee5-a7b8-436b-8685-ff90f478d369","Type":"ContainerStarted","Data":"a05a0b752591a5f60a65d5e38b85a4a29aefb23e1fcc29ee994d61c91735e5a3"}
Nov 04 02:23:09.379391 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:09.379242    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns" event={"ID":"4e50fee5-a7b8-436b-8685-ff90f478d369","Type":"ContainerStarted","Data":"92ec2fcfa8b6ad8b432df874eb09728524e691957d0a420b5b45f83d7ef92cf4"}
Nov 04 02:23:09.500812 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:09.500756    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns" podStartSLOduration=1.500736589 podStartE2EDuration="1.500736589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:23:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:23:09.499547694 +0000 UTC m=+1025.425770328" watchObservedRunningTime="2026-11-04 02:23:09.500736589 +0000 UTC m=+1025.426959223"
Nov 04 02:23:09.824665 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:09.824630    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:10.787907 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:10.787869    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:10.826588 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:10.826555    4969 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 04 02:23:10.826964 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:10.826607    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns" podUID="4e50fee5-a7b8-436b-8685-ff90f478d369" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 04 02:23:11.825176 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:11.825143    4969 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 04 02:23:11.825374 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:11.825195    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns" podUID="4e50fee5-a7b8-436b-8685-ff90f478d369" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 04 02:23:12.686216 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:12.686152    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx" podUID="49e51d64-9c71-4a5c-8ff3-8dd79cc6d487" containerName="payload-processing" containerID="cri-o://c6ed892083f3237b989d6b8d8e48745d437f07bfed053a82aa1c4c95d7f72039" gracePeriod=25
Nov 04 02:23:12.824563 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:12.824525    4969 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 04 02:23:12.824740 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:12.824586    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns" podUID="4e50fee5-a7b8-436b-8685-ff90f478d369" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 04 02:23:12.872117 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:12.868146    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt" podUID="cb250458-8451-45f4-a29b-22d4ccb929c1" containerName="payload-pre-processing" containerID="cri-o://b5b3c2b18c94a2864db43d71097bda1788e5d6711b6c63414cf905320a69e705" gracePeriod=25
Nov 04 02:23:13.504672 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:13.504594    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:13.825584 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:13.825550    4969 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 04 02:23:13.825983 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:13.825601    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns" podUID="4e50fee5-a7b8-436b-8685-ff90f478d369" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 04 02:23:14.513458 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:14.513417    4969 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx" podUID="49e51d64-9c71-4a5c-8ff3-8dd79cc6d487" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:23:14.513660 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:14.513429    4969 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt" podUID="cb250458-8451-45f4-a29b-22d4ccb929c1" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:23:14.825082 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:14.825001    4969 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 04 02:23:14.825256 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:14.825082    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns" podUID="4e50fee5-a7b8-436b-8685-ff90f478d369" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 04 02:23:15.780633 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:15.780582    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:15.825022 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:15.824984    4969 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 04 02:23:15.825238 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:15.825043    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns" podUID="4e50fee5-a7b8-436b-8685-ff90f478d369" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 04 02:23:16.828869 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:16.828833    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:16.829369 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:16.829212    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:16.829993 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:16.829970    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:23:19.377226 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:19.377190    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:20.550375 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:20.550334    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:20.724548 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:20.724514    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:23.748625 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:23.748592    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:24.513336 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:24.513292    4969 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt" podUID="cb250458-8451-45f4-a29b-22d4ccb929c1" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:23:24.513516 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:24.513294    4969 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx" podUID="49e51d64-9c71-4a5c-8ff3-8dd79cc6d487" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:23:24.874004 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:24.873913    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn"]
Nov 04 02:23:24.874484 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:24.874187    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn" podUID="8b999b27-4022-43c6-b758-5e36ef2a1776" containerName="maas-api" containerID="cri-o://68462f36be3c50f826bc4ed10709dd7950cf09bde70f42562c2e21ba2b0bdcd5" gracePeriod=30
Nov 04 02:23:25.022610 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:25.022573    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:25.489136 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:25.489089    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"]
Nov 04 02:23:25.620813 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:25.620783    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:26.146049 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:26.146018    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn"
Nov 04 02:23:26.241238 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:26.241162    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8b999b27-4022-43c6-b758-5e36ef2a1776-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b999b27-4022-43c6-b758-5e36ef2a1776-openshift-service-ca\") pod \"8b999b27-4022-43c6-b758-5e36ef2a1776\" (UID: \"8b999b27-4022-43c6-b758-5e36ef2a1776\") "
Nov 04 02:23:26.241477 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:26.241456    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b999b27-4022-43c6-b758-5e36ef2a1776-openshift-service-ca" pod "8b999b27-4022-43c6-b758-5e36ef2a1776" (UID: "8b999b27-4022-43c6-b758-5e36ef2a1776"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:23:26.342404 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:26.342375    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b999b27-4022-43c6-b758-5e36ef2a1776-kube-api-access-lmccw\" (UniqueName: \"kubernetes.io/projected/8b999b27-4022-43c6-b758-5e36ef2a1776-kube-api-access-lmccw\") pod \"8b999b27-4022-43c6-b758-5e36ef2a1776\" (UID: \"8b999b27-4022-43c6-b758-5e36ef2a1776\") "
Nov 04 02:23:26.342554 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:26.342420    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b999b27-4022-43c6-b758-5e36ef2a1776-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b999b27-4022-43c6-b758-5e36ef2a1776-maas-api-tls\") pod \"8b999b27-4022-43c6-b758-5e36ef2a1776\" (UID: \"8b999b27-4022-43c6-b758-5e36ef2a1776\") "
Nov 04 02:23:26.342554 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:26.342478    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b999b27-4022-43c6-b758-5e36ef2a1776-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b999b27-4022-43c6-b758-5e36ef2a1776-metrics-certs\") pod \"8b999b27-4022-43c6-b758-5e36ef2a1776\" (UID: \"8b999b27-4022-43c6-b758-5e36ef2a1776\") "
Nov 04 02:23:26.342893 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:26.342874    4969 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b999b27-4022-43c6-b758-5e36ef2a1776-openshift-service-ca\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:26.344653 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:26.344622    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b999b27-4022-43c6-b758-5e36ef2a1776-metrics-certs" pod "8b999b27-4022-43c6-b758-5e36ef2a1776" (UID: "8b999b27-4022-43c6-b758-5e36ef2a1776"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:23:26.344756 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:26.344682    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b999b27-4022-43c6-b758-5e36ef2a1776-maas-api-tls" pod "8b999b27-4022-43c6-b758-5e36ef2a1776" (UID: "8b999b27-4022-43c6-b758-5e36ef2a1776"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:23:26.344756 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:26.344717    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b999b27-4022-43c6-b758-5e36ef2a1776-kube-api-access-lmccw" pod "8b999b27-4022-43c6-b758-5e36ef2a1776" (UID: "8b999b27-4022-43c6-b758-5e36ef2a1776"). InnerVolumeSpecName "kube-api-access-lmccw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:23:26.444077 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:26.444053    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmccw\" (UniqueName: \"kubernetes.io/projected/8b999b27-4022-43c6-b758-5e36ef2a1776-kube-api-access-lmccw\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:26.444077 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:26.444079    4969 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b999b27-4022-43c6-b758-5e36ef2a1776-maas-api-tls\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:26.444242 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:26.444088    4969 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b999b27-4022-43c6-b758-5e36ef2a1776-metrics-certs\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:26.497475 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:26.497412    4969 generic.go:356] "Generic (PLEG): container finished" podID="8b999b27-4022-43c6-b758-5e36ef2a1776" containerID="68462f36be3c50f826bc4ed10709dd7950cf09bde70f42562c2e21ba2b0bdcd5" exitCode=0
Nov 04 02:23:26.497475 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:26.497460    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn"
Nov 04 02:23:26.497605 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:26.497471    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn" event={"ID":"8b999b27-4022-43c6-b758-5e36ef2a1776","Type":"ContainerDied","Data":"68462f36be3c50f826bc4ed10709dd7950cf09bde70f42562c2e21ba2b0bdcd5"}
Nov 04 02:23:26.497605 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:26.497521    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn" event={"ID":"8b999b27-4022-43c6-b758-5e36ef2a1776","Type":"ContainerDied","Data":"7b924b696773ba627c05453ac11616fab9bdec8979467f1c14113704ae40d094"}
Nov 04 02:23:26.497605 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:26.497552    4969 scope.go:122] "RemoveContainer" containerID="68462f36be3c50f826bc4ed10709dd7950cf09bde70f42562c2e21ba2b0bdcd5"
Nov 04 02:23:26.508006 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:26.507984    4969 scope.go:122] "RemoveContainer" containerID="68462f36be3c50f826bc4ed10709dd7950cf09bde70f42562c2e21ba2b0bdcd5"
Nov 04 02:23:26.508280 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:23:26.508261    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68462f36be3c50f826bc4ed10709dd7950cf09bde70f42562c2e21ba2b0bdcd5\": container with ID starting with 68462f36be3c50f826bc4ed10709dd7950cf09bde70f42562c2e21ba2b0bdcd5 not found: ID does not exist" containerID="68462f36be3c50f826bc4ed10709dd7950cf09bde70f42562c2e21ba2b0bdcd5"
Nov 04 02:23:26.508351 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:26.508291    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68462f36be3c50f826bc4ed10709dd7950cf09bde70f42562c2e21ba2b0bdcd5"} err="failed to get container status \"68462f36be3c50f826bc4ed10709dd7950cf09bde70f42562c2e21ba2b0bdcd5\": rpc error: code = NotFound desc = could not find container \"68462f36be3c50f826bc4ed10709dd7950cf09bde70f42562c2e21ba2b0bdcd5\": container with ID starting with 68462f36be3c50f826bc4ed10709dd7950cf09bde70f42562c2e21ba2b0bdcd5 not found: ID does not exist"
Nov 04 02:23:26.652993 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:26.652970    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:26.664015 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:26.663995    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn"]
Nov 04 02:23:26.667496 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:26.667467    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-e34b54-7cd5778bcb-c6vdn"]
Nov 04 02:23:26.840877 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:26.840849    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:27.016343 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:27.016313    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:28.202603 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:28.202567    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b999b27-4022-43c6-b758-5e36ef2a1776" path="/var/lib/kubelet/pods/8b999b27-4022-43c6-b758-5e36ef2a1776/volumes"
Nov 04 02:23:28.681239 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:28.681209    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"]
Nov 04 02:23:28.775692 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:28.775659    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:28.950661 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:28.950586    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:31.736080 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:31.736047    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:31.912513 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:31.912478    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:32.668550 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:32.668514    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:33.020552 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.020522    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx"
Nov 04 02:23:33.111539 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.111503    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/49e51d64-9c71-4a5c-8ff3-8dd79cc6d487-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49e51d64-9c71-4a5c-8ff3-8dd79cc6d487-plugins-config-volume\") pod \"49e51d64-9c71-4a5c-8ff3-8dd79cc6d487\" (UID: \"49e51d64-9c71-4a5c-8ff3-8dd79cc6d487\") "
Nov 04 02:23:33.111715 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.111685    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/49e51d64-9c71-4a5c-8ff3-8dd79cc6d487-kube-api-access-gt59x\" (UniqueName: \"kubernetes.io/projected/49e51d64-9c71-4a5c-8ff3-8dd79cc6d487-kube-api-access-gt59x\") pod \"49e51d64-9c71-4a5c-8ff3-8dd79cc6d487\" (UID: \"49e51d64-9c71-4a5c-8ff3-8dd79cc6d487\") "
Nov 04 02:23:33.112514 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.112432    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49e51d64-9c71-4a5c-8ff3-8dd79cc6d487-plugins-config-volume" pod "49e51d64-9c71-4a5c-8ff3-8dd79cc6d487" (UID: "49e51d64-9c71-4a5c-8ff3-8dd79cc6d487"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:23:33.113995 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.113970    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49e51d64-9c71-4a5c-8ff3-8dd79cc6d487-kube-api-access-gt59x" pod "49e51d64-9c71-4a5c-8ff3-8dd79cc6d487" (UID: "49e51d64-9c71-4a5c-8ff3-8dd79cc6d487"). InnerVolumeSpecName "kube-api-access-gt59x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:23:33.132553 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.132530    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt"
Nov 04 02:23:33.212844 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.212767    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cb250458-8451-45f4-a29b-22d4ccb929c1-kube-api-access-4xjqk\" (UniqueName: \"kubernetes.io/projected/cb250458-8451-45f4-a29b-22d4ccb929c1-kube-api-access-4xjqk\") pod \"cb250458-8451-45f4-a29b-22d4ccb929c1\" (UID: \"cb250458-8451-45f4-a29b-22d4ccb929c1\") "
Nov 04 02:23:33.212844 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.212840    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cb250458-8451-45f4-a29b-22d4ccb929c1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb250458-8451-45f4-a29b-22d4ccb929c1-plugins-config-volume\") pod \"cb250458-8451-45f4-a29b-22d4ccb929c1\" (UID: \"cb250458-8451-45f4-a29b-22d4ccb929c1\") "
Nov 04 02:23:33.213203 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.213182    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gt59x\" (UniqueName: \"kubernetes.io/projected/49e51d64-9c71-4a5c-8ff3-8dd79cc6d487-kube-api-access-gt59x\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:33.213274 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.213210    4969 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49e51d64-9c71-4a5c-8ff3-8dd79cc6d487-plugins-config-volume\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:33.213274 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.213229    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb250458-8451-45f4-a29b-22d4ccb929c1-plugins-config-volume" pod "cb250458-8451-45f4-a29b-22d4ccb929c1" (UID: "cb250458-8451-45f4-a29b-22d4ccb929c1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:23:33.214938 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.214920    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb250458-8451-45f4-a29b-22d4ccb929c1-kube-api-access-4xjqk" pod "cb250458-8451-45f4-a29b-22d4ccb929c1" (UID: "cb250458-8451-45f4-a29b-22d4ccb929c1"). InnerVolumeSpecName "kube-api-access-4xjqk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:23:33.314434 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.314398    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xjqk\" (UniqueName: \"kubernetes.io/projected/cb250458-8451-45f4-a29b-22d4ccb929c1-kube-api-access-4xjqk\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:33.314434 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.314435    4969 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb250458-8451-45f4-a29b-22d4ccb929c1-plugins-config-volume\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:33.539009 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.538972    4969 generic.go:356] "Generic (PLEG): container finished" podID="cb250458-8451-45f4-a29b-22d4ccb929c1" containerID="b5b3c2b18c94a2864db43d71097bda1788e5d6711b6c63414cf905320a69e705" exitCode=0
Nov 04 02:23:33.539235 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.539016    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt"
Nov 04 02:23:33.539235 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.539056    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt" event={"ID":"cb250458-8451-45f4-a29b-22d4ccb929c1","Type":"ContainerDied","Data":"b5b3c2b18c94a2864db43d71097bda1788e5d6711b6c63414cf905320a69e705"}
Nov 04 02:23:33.539235 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.539128    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt" event={"ID":"cb250458-8451-45f4-a29b-22d4ccb929c1","Type":"ContainerDied","Data":"aa3c31eac0ae68dcedb609a7e0d367b4d058aecf4b36d8b2dc0e183cf8319080"}
Nov 04 02:23:33.539235 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.539151    4969 scope.go:122] "RemoveContainer" containerID="b5b3c2b18c94a2864db43d71097bda1788e5d6711b6c63414cf905320a69e705"
Nov 04 02:23:33.540577 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.540556    4969 generic.go:356] "Generic (PLEG): container finished" podID="49e51d64-9c71-4a5c-8ff3-8dd79cc6d487" containerID="c6ed892083f3237b989d6b8d8e48745d437f07bfed053a82aa1c4c95d7f72039" exitCode=0
Nov 04 02:23:33.540657 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.540589    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx" event={"ID":"49e51d64-9c71-4a5c-8ff3-8dd79cc6d487","Type":"ContainerDied","Data":"c6ed892083f3237b989d6b8d8e48745d437f07bfed053a82aa1c4c95d7f72039"}
Nov 04 02:23:33.540657 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.540619    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx" event={"ID":"49e51d64-9c71-4a5c-8ff3-8dd79cc6d487","Type":"ContainerDied","Data":"0589e6df2c2a088d46a0e20daa4536c0a81484d3af7a548b2910d48e2c48afe3"}
Nov 04 02:23:33.540657 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.540625    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx"
Nov 04 02:23:33.549538 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.549519    4969 scope.go:122] "RemoveContainer" containerID="b5b3c2b18c94a2864db43d71097bda1788e5d6711b6c63414cf905320a69e705"
Nov 04 02:23:33.549917 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:23:33.549795    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5b3c2b18c94a2864db43d71097bda1788e5d6711b6c63414cf905320a69e705\": container with ID starting with b5b3c2b18c94a2864db43d71097bda1788e5d6711b6c63414cf905320a69e705 not found: ID does not exist" containerID="b5b3c2b18c94a2864db43d71097bda1788e5d6711b6c63414cf905320a69e705"
Nov 04 02:23:33.549917 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.549823    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5b3c2b18c94a2864db43d71097bda1788e5d6711b6c63414cf905320a69e705"} err="failed to get container status \"b5b3c2b18c94a2864db43d71097bda1788e5d6711b6c63414cf905320a69e705\": rpc error: code = NotFound desc = could not find container \"b5b3c2b18c94a2864db43d71097bda1788e5d6711b6c63414cf905320a69e705\": container with ID starting with b5b3c2b18c94a2864db43d71097bda1788e5d6711b6c63414cf905320a69e705 not found: ID does not exist"
Nov 04 02:23:33.549917 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.549839    4969 scope.go:122] "RemoveContainer" containerID="c6ed892083f3237b989d6b8d8e48745d437f07bfed053a82aa1c4c95d7f72039"
Nov 04 02:23:33.558703 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.558687    4969 scope.go:122] "RemoveContainer" containerID="c6ed892083f3237b989d6b8d8e48745d437f07bfed053a82aa1c4c95d7f72039"
Nov 04 02:23:33.558986 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:23:33.558969    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6ed892083f3237b989d6b8d8e48745d437f07bfed053a82aa1c4c95d7f72039\": container with ID starting with c6ed892083f3237b989d6b8d8e48745d437f07bfed053a82aa1c4c95d7f72039 not found: ID does not exist" containerID="c6ed892083f3237b989d6b8d8e48745d437f07bfed053a82aa1c4c95d7f72039"
Nov 04 02:23:33.559048 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.558996    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6ed892083f3237b989d6b8d8e48745d437f07bfed053a82aa1c4c95d7f72039"} err="failed to get container status \"c6ed892083f3237b989d6b8d8e48745d437f07bfed053a82aa1c4c95d7f72039\": rpc error: code = NotFound desc = could not find container \"c6ed892083f3237b989d6b8d8e48745d437f07bfed053a82aa1c4c95d7f72039\": container with ID starting with c6ed892083f3237b989d6b8d8e48745d437f07bfed053a82aa1c4c95d7f72039 not found: ID does not exist"
Nov 04 02:23:33.703263 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.703230    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx"]
Nov 04 02:23:33.709152 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.709127    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-e34b54-7f5dbdf5cb-992jx"]
Nov 04 02:23:33.817175 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.817087    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt"]
Nov 04 02:23:33.821663 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:33.821638    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-e34b54-79d79bdd59-5d2wt"]
Nov 04 02:23:34.204381 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:34.204294    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="49e51d64-9c71-4a5c-8ff3-8dd79cc6d487" path="/var/lib/kubelet/pods/49e51d64-9c71-4a5c-8ff3-8dd79cc6d487/volumes"
Nov 04 02:23:34.204959 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:34.204915    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cb250458-8451-45f4-a29b-22d4ccb929c1" path="/var/lib/kubelet/pods/cb250458-8451-45f4-a29b-22d4ccb929c1/volumes"
Nov 04 02:23:34.261275 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:23:34.261239    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:23:35.717925 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:35.717893    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:36.020236 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:36.020200    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:36.765541 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:36.765491    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:37.340342 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:37.340310    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:37.533172 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:37.533131    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:37.554439 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:37.554407    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2"]
Nov 04 02:23:37.659157 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:37.658969    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99"]
Nov 04 02:23:37.969259 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:37.969166    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:38.046945 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:38.046912    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"]
Nov 04 02:23:38.047194 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:38.047167    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4" podUID="6d2b50e4-af78-4a23-8a66-000bfd766bb3" containerName="istio-proxy" containerID="cri-o://487fc85a35ad834004f00cd7b0657ef139e09bee09aa4d46997f923d8e816594" gracePeriod=30
Nov 04 02:23:38.155339 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:38.155303    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:39.300107 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:39.300066    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:39.484113 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:39.484065    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:40.514716 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:40.514673    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl" podUID="92e2ae41-9ce7-47d9-915c-bd6d70ddd2de" containerName="main" containerID="cri-o://1f26e31f666e7675089a67a0f36f2fe72f0aae94bb59ae72facb9ac67de52e8c" gracePeriod=45
Nov 04 02:23:40.804298 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:40.804271    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"
Nov 04 02:23:40.891247 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:40.891218    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-dshm\" (UniqueName: \"kubernetes.io/empty-dir/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-dshm\") pod \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\" (UID: \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\") "
Nov 04 02:23:40.891404 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:40.891260    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-home\" (UniqueName: \"kubernetes.io/empty-dir/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-home\") pod \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\" (UID: \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\") "
Nov 04 02:23:40.891404 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:40.891286    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-model-cache\") pod \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\" (UID: \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\") "
Nov 04 02:23:40.891404 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:40.891318    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-tls-certs\" (UniqueName: \"kubernetes.io/secret/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-tls-certs\") pod \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\" (UID: \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\") "
Nov 04 02:23:40.891404 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:40.891368    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-kube-api-access-hfqs8\" (UniqueName: \"kubernetes.io/projected/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-kube-api-access-hfqs8\") pod \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\" (UID: \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\") "
Nov 04 02:23:40.891639 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:40.891407    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-tmp-dir\") pod \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\" (UID: \"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de\") "
Nov 04 02:23:40.891639 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:40.891532    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-model-cache" pod "92e2ae41-9ce7-47d9-915c-bd6d70ddd2de" (UID: "92e2ae41-9ce7-47d9-915c-bd6d70ddd2de"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:40.891639 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:40.891550    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-home" pod "92e2ae41-9ce7-47d9-915c-bd6d70ddd2de" (UID: "92e2ae41-9ce7-47d9-915c-bd6d70ddd2de"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:40.891769 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:40.891746    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-tmp-dir" pod "92e2ae41-9ce7-47d9-915c-bd6d70ddd2de" (UID: "92e2ae41-9ce7-47d9-915c-bd6d70ddd2de"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:40.891877 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:40.891856    4969 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-home\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:40.891955 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:40.891883    4969 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-model-cache\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:40.891955 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:40.891898    4969 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-tmp-dir\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:40.893691 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:40.893665    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-dshm" pod "92e2ae41-9ce7-47d9-915c-bd6d70ddd2de" (UID: "92e2ae41-9ce7-47d9-915c-bd6d70ddd2de"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:40.893793 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:40.893730    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-tls-certs" pod "92e2ae41-9ce7-47d9-915c-bd6d70ddd2de" (UID: "92e2ae41-9ce7-47d9-915c-bd6d70ddd2de"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:23:40.893793 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:40.893780    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-kube-api-access-hfqs8" pod "92e2ae41-9ce7-47d9-915c-bd6d70ddd2de" (UID: "92e2ae41-9ce7-47d9-915c-bd6d70ddd2de"). InnerVolumeSpecName "kube-api-access-hfqs8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:23:40.992588 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:40.992546    4969 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-dshm\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:40.992588 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:40.992585    4969 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-tls-certs\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:40.992765 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:40.992600    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfqs8\" (UniqueName: \"kubernetes.io/projected/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de-kube-api-access-hfqs8\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:41.524921 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.524888    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-p42b2-dzdbc"]
Nov 04 02:23:41.525404 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.525390    4969 cpu_manager.go:414] "containerMap: removing container" podUID="49e51d64-9c71-4a5c-8ff3-8dd79cc6d487" containerName="payload-processing"
Nov 04 02:23:41.525454 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.525405    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49e51d64-9c71-4a5c-8ff3-8dd79cc6d487" containerName="payload-processing"
Nov 04 02:23:41.525454 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.525416    4969 cpu_manager.go:414] "containerMap: removing container" podUID="cb250458-8451-45f4-a29b-22d4ccb929c1" containerName="payload-pre-processing"
Nov 04 02:23:41.525454 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.525422    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cb250458-8451-45f4-a29b-22d4ccb929c1" containerName="payload-pre-processing"
Nov 04 02:23:41.525454 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.525438    4969 cpu_manager.go:414] "containerMap: removing container" podUID="8b999b27-4022-43c6-b758-5e36ef2a1776" containerName="maas-api"
Nov 04 02:23:41.525454 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.525444    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b999b27-4022-43c6-b758-5e36ef2a1776" containerName="maas-api"
Nov 04 02:23:41.525604 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.525456    4969 cpu_manager.go:414] "containerMap: removing container" podUID="92e2ae41-9ce7-47d9-915c-bd6d70ddd2de" containerName="main"
Nov 04 02:23:41.525604 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.525462    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="92e2ae41-9ce7-47d9-915c-bd6d70ddd2de" containerName="main"
Nov 04 02:23:41.525604 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.525542    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="49e51d64-9c71-4a5c-8ff3-8dd79cc6d487" containerName="payload-processing"
Nov 04 02:23:41.525604 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.525549    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="cb250458-8451-45f4-a29b-22d4ccb929c1" containerName="payload-pre-processing"
Nov 04 02:23:41.525604 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.525557    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b999b27-4022-43c6-b758-5e36ef2a1776" containerName="maas-api"
Nov 04 02:23:41.525604 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.525563    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="92e2ae41-9ce7-47d9-915c-bd6d70ddd2de" containerName="main"
Nov 04 02:23:41.529053 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.529034    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-p42b2-dzdbc"
Nov 04 02:23:41.585880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.585855    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-p42b2-dzdbc"]
Nov 04 02:23:41.593291 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.593270    4969 generic.go:356] "Generic (PLEG): container finished" podID="92e2ae41-9ce7-47d9-915c-bd6d70ddd2de" containerID="1f26e31f666e7675089a67a0f36f2fe72f0aae94bb59ae72facb9ac67de52e8c" exitCode=0
Nov 04 02:23:41.593391 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.593337    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"
Nov 04 02:23:41.593440 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.593338    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl" event={"ID":"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de","Type":"ContainerDied","Data":"1f26e31f666e7675089a67a0f36f2fe72f0aae94bb59ae72facb9ac67de52e8c"}
Nov 04 02:23:41.593492 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.593471    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl" event={"ID":"92e2ae41-9ce7-47d9-915c-bd6d70ddd2de","Type":"ContainerDied","Data":"d296701a6a40726083a072ee92057bf9d93a5384120918cfa147d2509bf358c6"}
Nov 04 02:23:41.593534 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.593506    4969 scope.go:122] "RemoveContainer" containerID="1f26e31f666e7675089a67a0f36f2fe72f0aae94bb59ae72facb9ac67de52e8c"
Nov 04 02:23:41.597265 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.597239    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a49f2af-8cb1-42fe-a537-4723b2437491-trusted-ca-bundle\") pod \"periodic-gathering-p42b2-dzdbc\" (UID: \"1a49f2af-8cb1-42fe-a537-4723b2437491\") " pod="openshift-insights/periodic-gathering-p42b2-dzdbc"
Nov 04 02:23:41.597666 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.597327    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/1a49f2af-8cb1-42fe-a537-4723b2437491-archives-path\") pod \"periodic-gathering-p42b2-dzdbc\" (UID: \"1a49f2af-8cb1-42fe-a537-4723b2437491\") " pod="openshift-insights/periodic-gathering-p42b2-dzdbc"
Nov 04 02:23:41.597666 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.597501    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a49f2af-8cb1-42fe-a537-4723b2437491-service-ca-bundle\") pod \"periodic-gathering-p42b2-dzdbc\" (UID: \"1a49f2af-8cb1-42fe-a537-4723b2437491\") " pod="openshift-insights/periodic-gathering-p42b2-dzdbc"
Nov 04 02:23:41.597666 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.597553    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc9k5\" (UniqueName: \"kubernetes.io/projected/1a49f2af-8cb1-42fe-a537-4723b2437491-kube-api-access-rc9k5\") pod \"periodic-gathering-p42b2-dzdbc\" (UID: \"1a49f2af-8cb1-42fe-a537-4723b2437491\") " pod="openshift-insights/periodic-gathering-p42b2-dzdbc"
Nov 04 02:23:41.604858 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.604771    4969 scope.go:122] "RemoveContainer" containerID="1f26e31f666e7675089a67a0f36f2fe72f0aae94bb59ae72facb9ac67de52e8c"
Nov 04 02:23:41.605075 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:23:41.605050    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f26e31f666e7675089a67a0f36f2fe72f0aae94bb59ae72facb9ac67de52e8c\": container with ID starting with 1f26e31f666e7675089a67a0f36f2fe72f0aae94bb59ae72facb9ac67de52e8c not found: ID does not exist" containerID="1f26e31f666e7675089a67a0f36f2fe72f0aae94bb59ae72facb9ac67de52e8c"
Nov 04 02:23:41.605173 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.605080    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f26e31f666e7675089a67a0f36f2fe72f0aae94bb59ae72facb9ac67de52e8c"} err="failed to get container status \"1f26e31f666e7675089a67a0f36f2fe72f0aae94bb59ae72facb9ac67de52e8c\": rpc error: code = NotFound desc = could not find container \"1f26e31f666e7675089a67a0f36f2fe72f0aae94bb59ae72facb9ac67de52e8c\": container with ID starting with 1f26e31f666e7675089a67a0f36f2fe72f0aae94bb59ae72facb9ac67de52e8c not found: ID does not exist"
Nov 04 02:23:41.688262 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.688232    4969 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl" podUID="92e2ae41-9ce7-47d9-915c-bd6d70ddd2de" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 02:23:41.698857 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.698833    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc9k5\" (UniqueName: \"kubernetes.io/projected/1a49f2af-8cb1-42fe-a537-4723b2437491-kube-api-access-rc9k5\") pod \"periodic-gathering-p42b2-dzdbc\" (UID: \"1a49f2af-8cb1-42fe-a537-4723b2437491\") " pod="openshift-insights/periodic-gathering-p42b2-dzdbc"
Nov 04 02:23:41.698951 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.698886    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a49f2af-8cb1-42fe-a537-4723b2437491-trusted-ca-bundle\") pod \"periodic-gathering-p42b2-dzdbc\" (UID: \"1a49f2af-8cb1-42fe-a537-4723b2437491\") " pod="openshift-insights/periodic-gathering-p42b2-dzdbc"
Nov 04 02:23:41.698951 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.698908    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/1a49f2af-8cb1-42fe-a537-4723b2437491-archives-path\") pod \"periodic-gathering-p42b2-dzdbc\" (UID: \"1a49f2af-8cb1-42fe-a537-4723b2437491\") " pod="openshift-insights/periodic-gathering-p42b2-dzdbc"
Nov 04 02:23:41.699054 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.698978    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a49f2af-8cb1-42fe-a537-4723b2437491-service-ca-bundle\") pod \"periodic-gathering-p42b2-dzdbc\" (UID: \"1a49f2af-8cb1-42fe-a537-4723b2437491\") " pod="openshift-insights/periodic-gathering-p42b2-dzdbc"
Nov 04 02:23:41.699407 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.699386    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/1a49f2af-8cb1-42fe-a537-4723b2437491-archives-path\") pod \"periodic-gathering-p42b2-dzdbc\" (UID: \"1a49f2af-8cb1-42fe-a537-4723b2437491\") " pod="openshift-insights/periodic-gathering-p42b2-dzdbc"
Nov 04 02:23:41.699664 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.699645    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a49f2af-8cb1-42fe-a537-4723b2437491-service-ca-bundle\") pod \"periodic-gathering-p42b2-dzdbc\" (UID: \"1a49f2af-8cb1-42fe-a537-4723b2437491\") " pod="openshift-insights/periodic-gathering-p42b2-dzdbc"
Nov 04 02:23:41.699744 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.699677    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a49f2af-8cb1-42fe-a537-4723b2437491-trusted-ca-bundle\") pod \"periodic-gathering-p42b2-dzdbc\" (UID: \"1a49f2af-8cb1-42fe-a537-4723b2437491\") " pod="openshift-insights/periodic-gathering-p42b2-dzdbc"
Nov 04 02:23:41.754256 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.754232    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"]
Nov 04 02:23:41.757836 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.757805    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-9adedd/premium-w6-9adedd-kserve-645dd5fcf-62lnl"]
Nov 04 02:23:41.791503 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.791453    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc9k5\" (UniqueName: \"kubernetes.io/projected/1a49f2af-8cb1-42fe-a537-4723b2437491-kube-api-access-rc9k5\") pod \"periodic-gathering-p42b2-dzdbc\" (UID: \"1a49f2af-8cb1-42fe-a537-4723b2437491\") " pod="openshift-insights/periodic-gathering-p42b2-dzdbc"
Nov 04 02:23:41.813575 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.813550    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:41.841129 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:41.841081    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-p42b2-dzdbc"
Nov 04 02:23:42.122493 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:42.122399    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:42.204968 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:42.204932    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="92e2ae41-9ce7-47d9-915c-bd6d70ddd2de" path="/var/lib/kubelet/pods/92e2ae41-9ce7-47d9-915c-bd6d70ddd2de/volumes"
Nov 04 02:23:42.217470 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:42.217444    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-p42b2-dzdbc"]
Nov 04 02:23:42.218961 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:23:42.218936    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a49f2af_8cb1_42fe_a537_4723b2437491.slice/crio-0d27a6a2dda771d1702594a778c97b87cdc10ad73ce2986dd985f021b602a624 WatchSource:0}: Error finding container 0d27a6a2dda771d1702594a778c97b87cdc10ad73ce2986dd985f021b602a624: Status 404 returned error can't find the container with id 0d27a6a2dda771d1702594a778c97b87cdc10ad73ce2986dd985f021b602a624
Nov 04 02:23:42.555417 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:42.555379    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2" podUID="75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a" containerName="payload-processing" containerID="cri-o://c5130a3402ecc15c5869b76950d0649e932355bd4ac2e4fecfda251837d9b5c8" gracePeriod=25
Nov 04 02:23:42.601193 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:42.601165    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-p42b2-dzdbc" event={"ID":"1a49f2af-8cb1-42fe-a537-4723b2437491","Type":"ContainerStarted","Data":"ad5ac6daa13a6c0cdb9f103fe3af955ca4531298ca0334ba3f5e8f0b239442d2"}
Nov 04 02:23:42.601314 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:42.601214    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-p42b2-dzdbc" event={"ID":"1a49f2af-8cb1-42fe-a537-4723b2437491","Type":"ContainerStarted","Data":"0d27a6a2dda771d1702594a778c97b87cdc10ad73ce2986dd985f021b602a624"}
Nov 04 02:23:42.659304 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:42.659269    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99" podUID="afaa9b17-4098-417a-aa81-eceb2ca22183" containerName="payload-pre-processing" containerID="cri-o://49192ea6e0209f3953cc846c74febfdca10a77b87f88571430946db5c31795b0" gracePeriod=25
Nov 04 02:23:42.716405 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:42.716353    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-p42b2-dzdbc" podStartSLOduration=1.7163389279999999 podStartE2EDuration="1.716338928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:23:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:23:42.715546748 +0000 UTC m=+1058.641769382" watchObservedRunningTime="2026-11-04 02:23:42.716338928 +0000 UTC m=+1058.642561555"
Nov 04 02:23:42.818730 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:42.818705    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2"
Nov 04 02:23:42.911824 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:42.911794    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a-kube-api-access-pr26w\" (UniqueName: \"kubernetes.io/projected/75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a-kube-api-access-pr26w\") pod \"75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a\" (UID: \"75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a\") "
Nov 04 02:23:42.911973 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:42.911865    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a-plugins-config-volume\") pod \"75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a\" (UID: \"75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a\") "
Nov 04 02:23:42.912225 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:42.912196    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a-plugins-config-volume" pod "75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a" (UID: "75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:23:42.913791 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:42.913770    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a-kube-api-access-pr26w" pod "75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a" (UID: "75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a"). InnerVolumeSpecName "kube-api-access-pr26w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:23:43.013588 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.013553    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pr26w\" (UniqueName: \"kubernetes.io/projected/75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a-kube-api-access-pr26w\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:43.013588 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.013582    4969 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a-plugins-config-volume\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:43.320722 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.320664    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:23:43.418005 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.417976    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-credential-socket\") pod \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") "
Nov 04 02:23:43.418193 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.418037    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6d2b50e4-af78-4a23-8a66-000bfd766bb3-kube-api-access-pz5c2\" (UniqueName: \"kubernetes.io/projected/6d2b50e4-af78-4a23-8a66-000bfd766bb3-kube-api-access-pz5c2\") pod \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") "
Nov 04 02:23:43.418193 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.418086    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istio-data\") pod \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") "
Nov 04 02:23:43.418193 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.418183    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istiod-ca-cert\") pod \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") "
Nov 04 02:23:43.418322 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.418237    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istio-podinfo\") pod \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") "
Nov 04 02:23:43.418322 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.418310    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-workload-socket\") pod \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") "
Nov 04 02:23:43.418411 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.418349    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istio-token\" (UniqueName: \"kubernetes.io/projected/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istio-token\") pod \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") "
Nov 04 02:23:43.418411 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.418374    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-credential-socket" pod "6d2b50e4-af78-4a23-8a66-000bfd766bb3" (UID: "6d2b50e4-af78-4a23-8a66-000bfd766bb3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:43.418507 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.418384    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-workload-certs\") pod \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") "
Nov 04 02:23:43.418507 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.418450    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istio-data" pod "6d2b50e4-af78-4a23-8a66-000bfd766bb3" (UID: "6d2b50e4-af78-4a23-8a66-000bfd766bb3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:43.418507 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.418479    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istio-envoy\") pod \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\" (UID: \"6d2b50e4-af78-4a23-8a66-000bfd766bb3\") "
Nov 04 02:23:43.418691 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.418664    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-workload-certs" pod "6d2b50e4-af78-4a23-8a66-000bfd766bb3" (UID: "6d2b50e4-af78-4a23-8a66-000bfd766bb3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:43.418784 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.418707    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istiod-ca-cert" pod "6d2b50e4-af78-4a23-8a66-000bfd766bb3" (UID: "6d2b50e4-af78-4a23-8a66-000bfd766bb3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:23:43.418784 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.418751    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-workload-socket" pod "6d2b50e4-af78-4a23-8a66-000bfd766bb3" (UID: "6d2b50e4-af78-4a23-8a66-000bfd766bb3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:43.418959 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.418937    4969 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-credential-socket\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:43.418959 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.418959    4969 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istio-data\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:43.419165 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.418969    4969 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istiod-ca-cert\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:43.419165 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.418977    4969 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-workload-socket\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:43.419165 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.418986    4969 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-workload-certs\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:43.420545 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.420521    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istio-token" pod "6d2b50e4-af78-4a23-8a66-000bfd766bb3" (UID: "6d2b50e4-af78-4a23-8a66-000bfd766bb3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:23:43.420635 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.420562    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istio-podinfo" pod "6d2b50e4-af78-4a23-8a66-000bfd766bb3" (UID: "6d2b50e4-af78-4a23-8a66-000bfd766bb3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:23:43.420689 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.420673    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istio-envoy" pod "6d2b50e4-af78-4a23-8a66-000bfd766bb3" (UID: "6d2b50e4-af78-4a23-8a66-000bfd766bb3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:43.420728 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.420687    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d2b50e4-af78-4a23-8a66-000bfd766bb3-kube-api-access-pz5c2" pod "6d2b50e4-af78-4a23-8a66-000bfd766bb3" (UID: "6d2b50e4-af78-4a23-8a66-000bfd766bb3"). InnerVolumeSpecName "kube-api-access-pz5c2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:23:43.519847 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.519813    4969 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istio-token\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:43.519847 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.519842    4969 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istio-envoy\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:43.519847 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.519851    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pz5c2\" (UniqueName: \"kubernetes.io/projected/6d2b50e4-af78-4a23-8a66-000bfd766bb3-kube-api-access-pz5c2\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:43.520070 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.519860    4969 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d2b50e4-af78-4a23-8a66-000bfd766bb3-istio-podinfo\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:43.610128 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.610024    4969 generic.go:356] "Generic (PLEG): container finished" podID="75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a" containerID="c5130a3402ecc15c5869b76950d0649e932355bd4ac2e4fecfda251837d9b5c8" exitCode=0
Nov 04 02:23:43.610128 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.610067    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2"
Nov 04 02:23:43.610603 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.610131    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2" event={"ID":"75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a","Type":"ContainerDied","Data":"c5130a3402ecc15c5869b76950d0649e932355bd4ac2e4fecfda251837d9b5c8"}
Nov 04 02:23:43.610603 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.610187    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2" event={"ID":"75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a","Type":"ContainerDied","Data":"3a775e02e4c9329370a5c25ed023124cdd6a167bf5ff9064b4627cd1e590ae89"}
Nov 04 02:23:43.610603 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.610221    4969 scope.go:122] "RemoveContainer" containerID="c5130a3402ecc15c5869b76950d0649e932355bd4ac2e4fecfda251837d9b5c8"
Nov 04 02:23:43.611632 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.611609    4969 generic.go:356] "Generic (PLEG): container finished" podID="6d2b50e4-af78-4a23-8a66-000bfd766bb3" containerID="487fc85a35ad834004f00cd7b0657ef139e09bee09aa4d46997f923d8e816594" exitCode=0
Nov 04 02:23:43.611743 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.611660    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"
Nov 04 02:23:43.611796 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.611719    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4" event={"ID":"6d2b50e4-af78-4a23-8a66-000bfd766bb3","Type":"ContainerDied","Data":"487fc85a35ad834004f00cd7b0657ef139e09bee09aa4d46997f923d8e816594"}
Nov 04 02:23:43.611796 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.611775    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4" event={"ID":"6d2b50e4-af78-4a23-8a66-000bfd766bb3","Type":"ContainerDied","Data":"a168ced9b0a6e828ba1e1abfd8510d878bb69321343fbcb70d5fefee3acedf66"}
Nov 04 02:23:43.619519 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.619495    4969 scope.go:122] "RemoveContainer" containerID="c5130a3402ecc15c5869b76950d0649e932355bd4ac2e4fecfda251837d9b5c8"
Nov 04 02:23:43.619817 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:23:43.619793    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5130a3402ecc15c5869b76950d0649e932355bd4ac2e4fecfda251837d9b5c8\": container with ID starting with c5130a3402ecc15c5869b76950d0649e932355bd4ac2e4fecfda251837d9b5c8 not found: ID does not exist" containerID="c5130a3402ecc15c5869b76950d0649e932355bd4ac2e4fecfda251837d9b5c8"
Nov 04 02:23:43.619879 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.619831    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5130a3402ecc15c5869b76950d0649e932355bd4ac2e4fecfda251837d9b5c8"} err="failed to get container status \"c5130a3402ecc15c5869b76950d0649e932355bd4ac2e4fecfda251837d9b5c8\": rpc error: code = NotFound desc = could not find container \"c5130a3402ecc15c5869b76950d0649e932355bd4ac2e4fecfda251837d9b5c8\": container with ID starting with c5130a3402ecc15c5869b76950d0649e932355bd4ac2e4fecfda251837d9b5c8 not found: ID does not exist"
Nov 04 02:23:43.619879 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.619854    4969 scope.go:122] "RemoveContainer" containerID="487fc85a35ad834004f00cd7b0657ef139e09bee09aa4d46997f923d8e816594"
Nov 04 02:23:43.628995 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.628978    4969 scope.go:122] "RemoveContainer" containerID="487fc85a35ad834004f00cd7b0657ef139e09bee09aa4d46997f923d8e816594"
Nov 04 02:23:43.629273 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:23:43.629255    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"487fc85a35ad834004f00cd7b0657ef139e09bee09aa4d46997f923d8e816594\": container with ID starting with 487fc85a35ad834004f00cd7b0657ef139e09bee09aa4d46997f923d8e816594 not found: ID does not exist" containerID="487fc85a35ad834004f00cd7b0657ef139e09bee09aa4d46997f923d8e816594"
Nov 04 02:23:43.629338 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.629280    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"487fc85a35ad834004f00cd7b0657ef139e09bee09aa4d46997f923d8e816594"} err="failed to get container status \"487fc85a35ad834004f00cd7b0657ef139e09bee09aa4d46997f923d8e816594\": rpc error: code = NotFound desc = could not find container \"487fc85a35ad834004f00cd7b0657ef139e09bee09aa4d46997f923d8e816594\": container with ID starting with 487fc85a35ad834004f00cd7b0657ef139e09bee09aa4d46997f923d8e816594 not found: ID does not exist"
Nov 04 02:23:43.701575 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.701547    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg" podUID="cfcfd03f-e20c-47ef-b048-b6203c30f52c" containerName="main" containerID="cri-o://a1cb768c5792236200e47ae188f9c5b4a7a4bc8c8924a8f23438b77659f635f6" gracePeriod=45
Nov 04 02:23:43.784916 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.784886    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"]
Nov 04 02:23:43.795382 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.795358    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-e34b54-openshift-default-854f9f566c-zmkb4"]
Nov 04 02:23:43.938423 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.938388    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2"]
Nov 04 02:23:43.965919 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.965889    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-9adedd-786c78f544-mtdn2"]
Nov 04 02:23:43.971366 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.971344    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"
Nov 04 02:23:43.978839 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:43.978817    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt"]
Nov 04 02:23:44.023360 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.023322    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cfcfd03f-e20c-47ef-b048-b6203c30f52c-kube-api-access-9bb8t\" (UniqueName: \"kubernetes.io/projected/cfcfd03f-e20c-47ef-b048-b6203c30f52c-kube-api-access-9bb8t\") pod \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\" (UID: \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\") "
Nov 04 02:23:44.023531 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.023375    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfcfd03f-e20c-47ef-b048-b6203c30f52c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfcfd03f-e20c-47ef-b048-b6203c30f52c-dshm\") pod \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\" (UID: \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\") "
Nov 04 02:23:44.023531 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.023461    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cfcfd03f-e20c-47ef-b048-b6203c30f52c-tls-certs\" (UniqueName: \"kubernetes.io/secret/cfcfd03f-e20c-47ef-b048-b6203c30f52c-tls-certs\") pod \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\" (UID: \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\") "
Nov 04 02:23:44.023531 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.023508    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfcfd03f-e20c-47ef-b048-b6203c30f52c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfcfd03f-e20c-47ef-b048-b6203c30f52c-tmp-dir\") pod \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\" (UID: \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\") "
Nov 04 02:23:44.023699 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.023597    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfcfd03f-e20c-47ef-b048-b6203c30f52c-home\" (UniqueName: \"kubernetes.io/empty-dir/cfcfd03f-e20c-47ef-b048-b6203c30f52c-home\") pod \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\" (UID: \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\") "
Nov 04 02:23:44.023699 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.023635    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfcfd03f-e20c-47ef-b048-b6203c30f52c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfcfd03f-e20c-47ef-b048-b6203c30f52c-model-cache\") pod \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\" (UID: \"cfcfd03f-e20c-47ef-b048-b6203c30f52c\") "
Nov 04 02:23:44.023902 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.023875    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfcfd03f-e20c-47ef-b048-b6203c30f52c-tmp-dir" pod "cfcfd03f-e20c-47ef-b048-b6203c30f52c" (UID: "cfcfd03f-e20c-47ef-b048-b6203c30f52c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:44.023960 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.023901    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfcfd03f-e20c-47ef-b048-b6203c30f52c-home" pod "cfcfd03f-e20c-47ef-b048-b6203c30f52c" (UID: "cfcfd03f-e20c-47ef-b048-b6203c30f52c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:44.024030 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.024011    4969 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfcfd03f-e20c-47ef-b048-b6203c30f52c-tmp-dir\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:44.024030 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.024018    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfcfd03f-e20c-47ef-b048-b6203c30f52c-model-cache" pod "cfcfd03f-e20c-47ef-b048-b6203c30f52c" (UID: "cfcfd03f-e20c-47ef-b048-b6203c30f52c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:44.025798 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.025769    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfcfd03f-e20c-47ef-b048-b6203c30f52c-tls-certs" pod "cfcfd03f-e20c-47ef-b048-b6203c30f52c" (UID: "cfcfd03f-e20c-47ef-b048-b6203c30f52c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:23:44.025933 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.025821    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfcfd03f-e20c-47ef-b048-b6203c30f52c-kube-api-access-9bb8t" pod "cfcfd03f-e20c-47ef-b048-b6203c30f52c" (UID: "cfcfd03f-e20c-47ef-b048-b6203c30f52c"). InnerVolumeSpecName "kube-api-access-9bb8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:23:44.026030 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.026009    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfcfd03f-e20c-47ef-b048-b6203c30f52c-dshm" pod "cfcfd03f-e20c-47ef-b048-b6203c30f52c" (UID: "cfcfd03f-e20c-47ef-b048-b6203c30f52c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:23:44.043265 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.043243    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk"]
Nov 04 02:23:44.125320 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.125240    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bb8t\" (UniqueName: \"kubernetes.io/projected/cfcfd03f-e20c-47ef-b048-b6203c30f52c-kube-api-access-9bb8t\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:44.125320 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.125276    4969 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfcfd03f-e20c-47ef-b048-b6203c30f52c-dshm\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:44.125320 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.125290    4969 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfcfd03f-e20c-47ef-b048-b6203c30f52c-tls-certs\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:44.125320 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.125303    4969 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfcfd03f-e20c-47ef-b048-b6203c30f52c-home\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:44.125320 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.125317    4969 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfcfd03f-e20c-47ef-b048-b6203c30f52c-model-cache\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:44.202251 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.202223    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6d2b50e4-af78-4a23-8a66-000bfd766bb3" path="/var/lib/kubelet/pods/6d2b50e4-af78-4a23-8a66-000bfd766bb3/volumes"
Nov 04 02:23:44.202637 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.202623    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a" path="/var/lib/kubelet/pods/75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a/volumes"
Nov 04 02:23:44.348365 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.348331    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:44.533652 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.533624    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:44.619061 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.619033    4969 generic.go:356] "Generic (PLEG): container finished" podID="cfcfd03f-e20c-47ef-b048-b6203c30f52c" containerID="a1cb768c5792236200e47ae188f9c5b4a7a4bc8c8924a8f23438b77659f635f6" exitCode=0
Nov 04 02:23:44.619503 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.619079    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"
Nov 04 02:23:44.619503 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.619121    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg" event={"ID":"cfcfd03f-e20c-47ef-b048-b6203c30f52c","Type":"ContainerDied","Data":"a1cb768c5792236200e47ae188f9c5b4a7a4bc8c8924a8f23438b77659f635f6"}
Nov 04 02:23:44.619503 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.619156    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg" event={"ID":"cfcfd03f-e20c-47ef-b048-b6203c30f52c","Type":"ContainerDied","Data":"7ee28e8edfd5b9859b7c325494c6c331d3872fa621464f446ee355cd25b3d0cc"}
Nov 04 02:23:44.619503 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.619173    4969 scope.go:122] "RemoveContainer" containerID="a1cb768c5792236200e47ae188f9c5b4a7a4bc8c8924a8f23438b77659f635f6"
Nov 04 02:23:44.628023 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.628003    4969 scope.go:122] "RemoveContainer" containerID="a1cb768c5792236200e47ae188f9c5b4a7a4bc8c8924a8f23438b77659f635f6"
Nov 04 02:23:44.628287 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:23:44.628271    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1cb768c5792236200e47ae188f9c5b4a7a4bc8c8924a8f23438b77659f635f6\": container with ID starting with a1cb768c5792236200e47ae188f9c5b4a7a4bc8c8924a8f23438b77659f635f6 not found: ID does not exist" containerID="a1cb768c5792236200e47ae188f9c5b4a7a4bc8c8924a8f23438b77659f635f6"
Nov 04 02:23:44.628353 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.628296    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1cb768c5792236200e47ae188f9c5b4a7a4bc8c8924a8f23438b77659f635f6"} err="failed to get container status \"a1cb768c5792236200e47ae188f9c5b4a7a4bc8c8924a8f23438b77659f635f6\": rpc error: code = NotFound desc = could not find container \"a1cb768c5792236200e47ae188f9c5b4a7a4bc8c8924a8f23438b77659f635f6\": container with ID starting with a1cb768c5792236200e47ae188f9c5b4a7a4bc8c8924a8f23438b77659f635f6 not found: ID does not exist"
Nov 04 02:23:44.783480 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.783449    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"]
Nov 04 02:23:44.788056 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.787993    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-dd5d18/distinct-2-w5-dd5d18-kserve-6d48dc495f-l5fsg"]
Nov 04 02:23:44.820255 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:44.820220    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:46.206332 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:46.206291    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cfcfd03f-e20c-47ef-b048-b6203c30f52c" path="/var/lib/kubelet/pods/cfcfd03f-e20c-47ef-b048-b6203c30f52c/volumes"
Nov 04 02:23:46.513509 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:46.513476    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs"]
Nov 04 02:23:46.513726 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:46.513704    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs" podUID="269fe5b3-6355-4cdc-a33a-fc4bfe897a79" containerName="maas-api" containerID="cri-o://7275ca47f22b120ffcbe01eb6269bb0bc8579392a40fba134996cb809346946e" gracePeriod=30
Nov 04 02:23:46.636801 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:46.636772    4969 generic.go:356] "Generic (PLEG): container finished" podID="269fe5b3-6355-4cdc-a33a-fc4bfe897a79" containerID="7275ca47f22b120ffcbe01eb6269bb0bc8579392a40fba134996cb809346946e" exitCode=0
Nov 04 02:23:46.636941 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:46.636848    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs" event={"ID":"269fe5b3-6355-4cdc-a33a-fc4bfe897a79","Type":"ContainerDied","Data":"7275ca47f22b120ffcbe01eb6269bb0bc8579392a40fba134996cb809346946e"}
Nov 04 02:23:46.656880 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:46.656853    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:46.774155 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:46.774104    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs"
Nov 04 02:23:46.849489 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:46.849457    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-openshift-service-ca\") pod \"269fe5b3-6355-4cdc-a33a-fc4bfe897a79\" (UID: \"269fe5b3-6355-4cdc-a33a-fc4bfe897a79\") "
Nov 04 02:23:46.849648 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:46.849522    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-metrics-certs\" (UniqueName: \"kubernetes.io/secret/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-metrics-certs\") pod \"269fe5b3-6355-4cdc-a33a-fc4bfe897a79\" (UID: \"269fe5b3-6355-4cdc-a33a-fc4bfe897a79\") "
Nov 04 02:23:46.849648 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:46.849620    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-maas-api-tls\") pod \"269fe5b3-6355-4cdc-a33a-fc4bfe897a79\" (UID: \"269fe5b3-6355-4cdc-a33a-fc4bfe897a79\") "
Nov 04 02:23:46.849779 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:46.849658    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-kube-api-access-v9qc4\" (UniqueName: \"kubernetes.io/projected/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-kube-api-access-v9qc4\") pod \"269fe5b3-6355-4cdc-a33a-fc4bfe897a79\" (UID: \"269fe5b3-6355-4cdc-a33a-fc4bfe897a79\") "
Nov 04 02:23:46.849858 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:46.849825    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-openshift-service-ca" pod "269fe5b3-6355-4cdc-a33a-fc4bfe897a79" (UID: "269fe5b3-6355-4cdc-a33a-fc4bfe897a79"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:23:46.851612 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:46.851589    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-maas-api-tls" pod "269fe5b3-6355-4cdc-a33a-fc4bfe897a79" (UID: "269fe5b3-6355-4cdc-a33a-fc4bfe897a79"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:23:46.851727 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:46.851711    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-metrics-certs" pod "269fe5b3-6355-4cdc-a33a-fc4bfe897a79" (UID: "269fe5b3-6355-4cdc-a33a-fc4bfe897a79"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:23:46.851828 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:46.851814    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-kube-api-access-v9qc4" pod "269fe5b3-6355-4cdc-a33a-fc4bfe897a79" (UID: "269fe5b3-6355-4cdc-a33a-fc4bfe897a79"). InnerVolumeSpecName "kube-api-access-v9qc4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:23:46.951601 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:46.951557    4969 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-maas-api-tls\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:46.951601 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:46.951603    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9qc4\" (UniqueName: \"kubernetes.io/projected/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-kube-api-access-v9qc4\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:46.951790 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:46.951617    4969 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-openshift-service-ca\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:46.951790 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:46.951626    4969 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/269fe5b3-6355-4cdc-a33a-fc4bfe897a79-metrics-certs\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:23:47.644015 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:47.643987    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs"
Nov 04 02:23:47.644461 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:47.644027    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs" event={"ID":"269fe5b3-6355-4cdc-a33a-fc4bfe897a79","Type":"ContainerDied","Data":"7e8b1296090622f5b4cd2a6562ebedf6cffac53342d623c725745b53ca55119f"}
Nov 04 02:23:47.644461 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:47.644104    4969 scope.go:122] "RemoveContainer" containerID="7275ca47f22b120ffcbe01eb6269bb0bc8579392a40fba134996cb809346946e"
Nov 04 02:23:47.828832 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:47.828797    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs"]
Nov 04 02:23:47.833439 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:47.833415    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-bggcs"]
Nov 04 02:23:47.959411 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:47.959314    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:48.134018 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:48.133981    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:48.202924 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:48.202890    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="269fe5b3-6355-4cdc-a33a-fc4bfe897a79" path="/var/lib/kubelet/pods/269fe5b3-6355-4cdc-a33a-fc4bfe897a79/volumes"
Nov 04 02:23:48.328483 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:48.328456    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:48.808883 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:48.808841    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:48.979124 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:48.979068    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt" podUID="6e4b23e2-526a-464c-bb05-2f156da5652b" containerName="payload-processing" containerID="cri-o://0a4f86cf1106335275af3546e544778829c13969a52c56675e077dd5af200500" gracePeriod=25
Nov 04 02:23:49.043836 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:49.043799    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk" podUID="62b73809-c52f-4c4a-915d-f03e77df2068" containerName="payload-pre-processing" containerID="cri-o://5dd339c514d916178d34e74a1e92edf5ca7dec1d38b16275bbb7199b196a27eb" gracePeriod=25
Nov 04 02:23:49.552592 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:49.552553    4969 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99" podUID="afaa9b17-4098-417a-aa81-eceb2ca22183" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:23:51.009942 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:51.009910    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:51.312490 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:51.312405    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:51.489947 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:51.489912    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:53.722442 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:53.722403    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg"]
Nov 04 02:23:53.722939 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:53.722887    4969 cpu_manager.go:414] "containerMap: removing container" podUID="75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a" containerName="payload-processing"
Nov 04 02:23:53.722939 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:53.722898    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a" containerName="payload-processing"
Nov 04 02:23:53.722939 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:53.722908    4969 cpu_manager.go:414] "containerMap: removing container" podUID="269fe5b3-6355-4cdc-a33a-fc4bfe897a79" containerName="maas-api"
Nov 04 02:23:53.722939 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:53.722914    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="269fe5b3-6355-4cdc-a33a-fc4bfe897a79" containerName="maas-api"
Nov 04 02:23:53.722939 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:53.722922    4969 cpu_manager.go:414] "containerMap: removing container" podUID="cfcfd03f-e20c-47ef-b048-b6203c30f52c" containerName="main"
Nov 04 02:23:53.722939 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:53.722928    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cfcfd03f-e20c-47ef-b048-b6203c30f52c" containerName="main"
Nov 04 02:23:53.722939 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:53.722935    4969 cpu_manager.go:414] "containerMap: removing container" podUID="6d2b50e4-af78-4a23-8a66-000bfd766bb3" containerName="istio-proxy"
Nov 04 02:23:53.722939 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:53.722940    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6d2b50e4-af78-4a23-8a66-000bfd766bb3" containerName="istio-proxy"
Nov 04 02:23:53.723439 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:53.723014    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="75a1aa0a-2af0-4dc7-8a0c-8e0ef2a9871a" containerName="payload-processing"
Nov 04 02:23:53.723439 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:53.723021    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="269fe5b3-6355-4cdc-a33a-fc4bfe897a79" containerName="maas-api"
Nov 04 02:23:53.723439 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:53.723027    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="cfcfd03f-e20c-47ef-b048-b6203c30f52c" containerName="main"
Nov 04 02:23:53.723439 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:53.723031    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="6d2b50e4-af78-4a23-8a66-000bfd766bb3" containerName="istio-proxy"
Nov 04 02:23:53.726370 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:53.726348    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg"
Nov 04 02:23:53.780088 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:53.780059    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-9adedd-serving-cert\""
Nov 04 02:23:53.839574 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:53.839534    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg\" (UID: \"ea0b94ed-20c6-4b3a-abd6-e39992324ec6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg"
Nov 04 02:23:53.839783 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:53.839765    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffjlw\" (UniqueName: \"kubernetes.io/projected/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-kube-api-access-ffjlw\") pod \"maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg\" (UID: \"ea0b94ed-20c6-4b3a-abd6-e39992324ec6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg"
Nov 04 02:23:53.839968 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:53.839955    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-maas-api-tls\") pod \"maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg\" (UID: \"ea0b94ed-20c6-4b3a-abd6-e39992324ec6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg"
Nov 04 02:23:53.840162 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:53.840141    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-metrics-certs\") pod \"maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg\" (UID: \"ea0b94ed-20c6-4b3a-abd6-e39992324ec6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg"
Nov 04 02:23:53.851617 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:53.851584    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg"]
Nov 04 02:23:53.911042 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:53.911004    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:53.960140 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:53.956258    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-maas-api-tls\") pod \"maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg\" (UID: \"ea0b94ed-20c6-4b3a-abd6-e39992324ec6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg"
Nov 04 02:23:53.960140 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:53.956352    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-metrics-certs\") pod \"maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg\" (UID: \"ea0b94ed-20c6-4b3a-abd6-e39992324ec6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg"
Nov 04 02:23:53.960140 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:53.956401    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg\" (UID: \"ea0b94ed-20c6-4b3a-abd6-e39992324ec6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg"
Nov 04 02:23:53.960140 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:53.956428    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffjlw\" (UniqueName: \"kubernetes.io/projected/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-kube-api-access-ffjlw\") pod \"maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg\" (UID: \"ea0b94ed-20c6-4b3a-abd6-e39992324ec6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg"
Nov 04 02:23:53.960140 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:23:53.956903    4969 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-serving-cert: secret "maas-api-e2e-worker-w6-9adedd-serving-cert" not found
Nov 04 02:23:53.960140 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:23:53.956975    4969 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-maas-api-tls podName:ea0b94ed-20c6-4b3a-abd6-e39992324ec6 nodeName:}" failed. No retries permitted until 2026-11-04 02:23:54.456956064 +0000 UTC m=+1070.383178681 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-maas-api-tls") pod "maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg" (UID: "ea0b94ed-20c6-4b3a-abd6-e39992324ec6") : secret "maas-api-e2e-worker-w6-9adedd-serving-cert" not found
Nov 04 02:23:53.960140 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:53.958269    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg\" (UID: \"ea0b94ed-20c6-4b3a-abd6-e39992324ec6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg"
Nov 04 02:23:54.006162 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:53.997463    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-metrics-certs\") pod \"maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg\" (UID: \"ea0b94ed-20c6-4b3a-abd6-e39992324ec6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg"
Nov 04 02:23:54.053741 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:54.053697    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffjlw\" (UniqueName: \"kubernetes.io/projected/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-kube-api-access-ffjlw\") pod \"maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg\" (UID: \"ea0b94ed-20c6-4b3a-abd6-e39992324ec6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg"
Nov 04 02:23:54.103300 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:54.103238    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:54.283461 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:54.283428    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:54.463790 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:54.463742    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-maas-api-tls\") pod \"maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg\" (UID: \"ea0b94ed-20c6-4b3a-abd6-e39992324ec6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg"
Nov 04 02:23:54.478477 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:54.478447    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-maas-api-tls\") pod \"maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg\" (UID: \"ea0b94ed-20c6-4b3a-abd6-e39992324ec6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg"
Nov 04 02:23:54.639122 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:54.639034    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg"
Nov 04 02:23:55.025675 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:55.025467    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg"]
Nov 04 02:23:55.028388 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:23:55.028346    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea0b94ed_20c6_4b3a_abd6_e39992324ec6.slice/crio-0c2af3a102492280f0c5fc7909285503895dd4460f83c39a938a561b1f896a67 WatchSource:0}: Error finding container 0c2af3a102492280f0c5fc7909285503895dd4460f83c39a938a561b1f896a67: Status 404 returned error can't find the container with id 0c2af3a102492280f0c5fc7909285503895dd4460f83c39a938a561b1f896a67
Nov 04 02:23:55.030285 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:55.030266    4969 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 02:23:55.700000 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:55.699956    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg" event={"ID":"ea0b94ed-20c6-4b3a-abd6-e39992324ec6","Type":"ContainerStarted","Data":"0c2af3a102492280f0c5fc7909285503895dd4460f83c39a938a561b1f896a67"}
Nov 04 02:23:56.708774 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:56.708733    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg" event={"ID":"ea0b94ed-20c6-4b3a-abd6-e39992324ec6","Type":"ContainerStarted","Data":"450b37f4b7b7782541cb836bd4ae4e4fd1e424f369f3127df0e7c7afbd1d4a84"}
Nov 04 02:23:56.709275 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:56.708906    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg"
Nov 04 02:23:56.830872 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:56.830804    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg" podStartSLOduration=2.922649569 podStartE2EDuration="3.830786526s" totalImagesPullingTime="908.136957ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:23:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:23:56.830420875 +0000 UTC m=+1072.756643508" watchObservedRunningTime="2026-11-04 02:23:56.830786526 +0000 UTC m=+1072.757009158"
Nov 04 02:23:57.051032 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:57.050999    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:57.538565 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:57.538517    4969 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk" podUID="62b73809-c52f-4c4a-915d-f03e77df2068" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:23:57.538799 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:57.538760    4969 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt" podUID="6e4b23e2-526a-464c-bb05-2f156da5652b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:23:58.836043 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:58.836007    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:23:59.552815 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:23:59.552776    4969 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99" podUID="afaa9b17-4098-417a-aa81-eceb2ca22183" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:24:00.036871 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:00.036841    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:02.719452 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:02.719417    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg"
Nov 04 02:24:02.978627 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:02.978599    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:02.987641 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:02.987617    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99"
Nov 04 02:24:03.045397 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:03.045365    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/afaa9b17-4098-417a-aa81-eceb2ca22183-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afaa9b17-4098-417a-aa81-eceb2ca22183-plugins-config-volume\") pod \"afaa9b17-4098-417a-aa81-eceb2ca22183\" (UID: \"afaa9b17-4098-417a-aa81-eceb2ca22183\") "
Nov 04 02:24:03.045547 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:03.045436    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/afaa9b17-4098-417a-aa81-eceb2ca22183-kube-api-access-9lkr7\" (UniqueName: \"kubernetes.io/projected/afaa9b17-4098-417a-aa81-eceb2ca22183-kube-api-access-9lkr7\") pod \"afaa9b17-4098-417a-aa81-eceb2ca22183\" (UID: \"afaa9b17-4098-417a-aa81-eceb2ca22183\") "
Nov 04 02:24:03.045823 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:03.045797    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afaa9b17-4098-417a-aa81-eceb2ca22183-plugins-config-volume" pod "afaa9b17-4098-417a-aa81-eceb2ca22183" (UID: "afaa9b17-4098-417a-aa81-eceb2ca22183"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:24:03.047723 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:03.047694    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afaa9b17-4098-417a-aa81-eceb2ca22183-kube-api-access-9lkr7" pod "afaa9b17-4098-417a-aa81-eceb2ca22183" (UID: "afaa9b17-4098-417a-aa81-eceb2ca22183"). InnerVolumeSpecName "kube-api-access-9lkr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:24:03.150137 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:03.146488    4969 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afaa9b17-4098-417a-aa81-eceb2ca22183-plugins-config-volume\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:03.150137 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:03.146524    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lkr7\" (UniqueName: \"kubernetes.io/projected/afaa9b17-4098-417a-aa81-eceb2ca22183-kube-api-access-9lkr7\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:03.171444 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:03.171390    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:03.762249 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:03.762211    4969 generic.go:356] "Generic (PLEG): container finished" podID="afaa9b17-4098-417a-aa81-eceb2ca22183" containerID="49192ea6e0209f3953cc846c74febfdca10a77b87f88571430946db5c31795b0" exitCode=0
Nov 04 02:24:03.762692 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:03.762260    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99"
Nov 04 02:24:03.762692 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:03.762302    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99" event={"ID":"afaa9b17-4098-417a-aa81-eceb2ca22183","Type":"ContainerDied","Data":"49192ea6e0209f3953cc846c74febfdca10a77b87f88571430946db5c31795b0"}
Nov 04 02:24:03.762692 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:03.762368    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99" event={"ID":"afaa9b17-4098-417a-aa81-eceb2ca22183","Type":"ContainerDied","Data":"a479af58cf99b26a2b90762db9adf7c8c244fc1ce4b0ef98082b1d66a9b90804"}
Nov 04 02:24:03.762692 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:03.762404    4969 scope.go:122] "RemoveContainer" containerID="49192ea6e0209f3953cc846c74febfdca10a77b87f88571430946db5c31795b0"
Nov 04 02:24:03.772273 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:03.772252    4969 scope.go:122] "RemoveContainer" containerID="49192ea6e0209f3953cc846c74febfdca10a77b87f88571430946db5c31795b0"
Nov 04 02:24:03.772540 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:24:03.772521    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49192ea6e0209f3953cc846c74febfdca10a77b87f88571430946db5c31795b0\": container with ID starting with 49192ea6e0209f3953cc846c74febfdca10a77b87f88571430946db5c31795b0 not found: ID does not exist" containerID="49192ea6e0209f3953cc846c74febfdca10a77b87f88571430946db5c31795b0"
Nov 04 02:24:03.772602 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:03.772552    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49192ea6e0209f3953cc846c74febfdca10a77b87f88571430946db5c31795b0"} err="failed to get container status \"49192ea6e0209f3953cc846c74febfdca10a77b87f88571430946db5c31795b0\": rpc error: code = NotFound desc = could not find container \"49192ea6e0209f3953cc846c74febfdca10a77b87f88571430946db5c31795b0\": container with ID starting with 49192ea6e0209f3953cc846c74febfdca10a77b87f88571430946db5c31795b0 not found: ID does not exist"
Nov 04 02:24:03.832037 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:03.832013    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:03.925293 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:03.925260    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99"]
Nov 04 02:24:03.974694 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:03.974669    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-9adedd-7f44c969c9-7cc99"]
Nov 04 02:24:04.005428 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:04.005402    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:04.201435 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:04.201368    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="afaa9b17-4098-417a-aa81-eceb2ca22183" path="/var/lib/kubelet/pods/afaa9b17-4098-417a-aa81-eceb2ca22183/volumes"
Nov 04 02:24:04.261019 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:24:04.260990    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:24:05.813654 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:05.813622    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:05.992947 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:05.992910    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:07.462512 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:07.462482    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:07.539043 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:07.539003    4969 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk" podUID="62b73809-c52f-4c4a-915d-f03e77df2068" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:24:07.539183 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:07.539003    4969 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt" podUID="6e4b23e2-526a-464c-bb05-2f156da5652b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 02:24:08.086724 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.086688    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv"]
Nov 04 02:24:08.086934 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.086891    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv" podUID="bccedafc-3320-4d2c-9a1f-a38dd01adac2" containerName="maas-api" containerID="cri-o://446063e0152795fdfbbb12d23281e5f54a35598eabb6ea2007e0ad14d90c51ca" gracePeriod=30
Nov 04 02:24:08.228894 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.228861    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:08.682685 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.682653    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"]
Nov 04 02:24:08.683203 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.683185    4969 cpu_manager.go:414] "containerMap: removing container" podUID="afaa9b17-4098-417a-aa81-eceb2ca22183" containerName="payload-pre-processing"
Nov 04 02:24:08.683248 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.683203    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="afaa9b17-4098-417a-aa81-eceb2ca22183" containerName="payload-pre-processing"
Nov 04 02:24:08.683303 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.683295    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="afaa9b17-4098-417a-aa81-eceb2ca22183" containerName="payload-pre-processing"
Nov 04 02:24:08.688365 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.688344    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:08.787745 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.787716    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"]
Nov 04 02:24:08.804713 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.804678    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-credential-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:08.804877 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.804723    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ba3e718-6552-472a-8002-259a7ffc4b89-istio-token\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:08.804948 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.804911    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfm2j\" (UniqueName: \"kubernetes.io/projected/0ba3e718-6552-472a-8002-259a7ffc4b89-kube-api-access-tfm2j\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:08.805003 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.804965    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-istio-envoy\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:08.805057 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.805034    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-istio-data\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:08.805162 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.805065    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ba3e718-6552-472a-8002-259a7ffc4b89-istiod-ca-cert\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:08.805304 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.805277    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-workload-certs\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:08.805422 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.805330    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ba3e718-6552-472a-8002-259a7ffc4b89-istio-podinfo\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:08.805422 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.805361    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-workload-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:08.906533 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.906499    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-workload-certs\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:08.906699 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.906543    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ba3e718-6552-472a-8002-259a7ffc4b89-istio-podinfo\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:08.906699 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.906576    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-workload-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:08.906699 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.906607    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-credential-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:08.906699 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.906630    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ba3e718-6552-472a-8002-259a7ffc4b89-istio-token\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:08.906907 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.906713    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfm2j\" (UniqueName: \"kubernetes.io/projected/0ba3e718-6552-472a-8002-259a7ffc4b89-kube-api-access-tfm2j\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:08.906907 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.906752    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-istio-envoy\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:08.906907 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.906780    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-istio-data\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:08.906907 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.906808    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ba3e718-6552-472a-8002-259a7ffc4b89-istiod-ca-cert\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:08.907141 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.906973    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-workload-certs\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:08.907141 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.907014    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-workload-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:08.907141 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.907082    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-credential-socket\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:08.907271 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.907254    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-istio-data\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:08.907506 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.907488    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ba3e718-6552-472a-8002-259a7ffc4b89-istiod-ca-cert\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:08.908921 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.908899    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-istio-envoy\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:08.909192 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.909160    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ba3e718-6552-472a-8002-259a7ffc4b89-istio-podinfo\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:08.967645 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.967575    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ba3e718-6552-472a-8002-259a7ffc4b89-istio-token\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:08.967763 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:08.967694    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfm2j\" (UniqueName: \"kubernetes.io/projected/0ba3e718-6552-472a-8002-259a7ffc4b89-kube-api-access-tfm2j\") pod \"e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") " pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:09.001367 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.001344    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:09.311138 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.310978    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt"
Nov 04 02:24:09.343194 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.343162    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk"
Nov 04 02:24:09.402141 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.402078    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv"
Nov 04 02:24:09.416350 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.416317    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/62b73809-c52f-4c4a-915d-f03e77df2068-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62b73809-c52f-4c4a-915d-f03e77df2068-plugins-config-volume\") pod \"62b73809-c52f-4c4a-915d-f03e77df2068\" (UID: \"62b73809-c52f-4c4a-915d-f03e77df2068\") "
Nov 04 02:24:09.416491 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.416372    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e4b23e2-526a-464c-bb05-2f156da5652b-kube-api-access-w7nmf\" (UniqueName: \"kubernetes.io/projected/6e4b23e2-526a-464c-bb05-2f156da5652b-kube-api-access-w7nmf\") pod \"6e4b23e2-526a-464c-bb05-2f156da5652b\" (UID: \"6e4b23e2-526a-464c-bb05-2f156da5652b\") "
Nov 04 02:24:09.416491 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.416413    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6e4b23e2-526a-464c-bb05-2f156da5652b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e4b23e2-526a-464c-bb05-2f156da5652b-plugins-config-volume\") pod \"6e4b23e2-526a-464c-bb05-2f156da5652b\" (UID: \"6e4b23e2-526a-464c-bb05-2f156da5652b\") "
Nov 04 02:24:09.416602 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.416526    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/62b73809-c52f-4c4a-915d-f03e77df2068-kube-api-access-r8ct6\" (UniqueName: \"kubernetes.io/projected/62b73809-c52f-4c4a-915d-f03e77df2068-kube-api-access-r8ct6\") pod \"62b73809-c52f-4c4a-915d-f03e77df2068\" (UID: \"62b73809-c52f-4c4a-915d-f03e77df2068\") "
Nov 04 02:24:09.416909 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.416887    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e4b23e2-526a-464c-bb05-2f156da5652b-plugins-config-volume" pod "6e4b23e2-526a-464c-bb05-2f156da5652b" (UID: "6e4b23e2-526a-464c-bb05-2f156da5652b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:24:09.416964 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.416892    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62b73809-c52f-4c4a-915d-f03e77df2068-plugins-config-volume" pod "62b73809-c52f-4c4a-915d-f03e77df2068" (UID: "62b73809-c52f-4c4a-915d-f03e77df2068"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:24:09.418740 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.418715    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62b73809-c52f-4c4a-915d-f03e77df2068-kube-api-access-r8ct6" pod "62b73809-c52f-4c4a-915d-f03e77df2068" (UID: "62b73809-c52f-4c4a-915d-f03e77df2068"). InnerVolumeSpecName "kube-api-access-r8ct6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:24:09.418841 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.418763    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e4b23e2-526a-464c-bb05-2f156da5652b-kube-api-access-w7nmf" pod "6e4b23e2-526a-464c-bb05-2f156da5652b" (UID: "6e4b23e2-526a-464c-bb05-2f156da5652b"). InnerVolumeSpecName "kube-api-access-w7nmf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:24:09.436324 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.434237    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"]
Nov 04 02:24:09.436324 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:24:09.435584    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ba3e718_6552_472a_8002_259a7ffc4b89.slice/crio-b9915ee470f8cb6cf67614a28a28886d45f197bfb51f9f4c335c46540dd43ab5 WatchSource:0}: Error finding container b9915ee470f8cb6cf67614a28a28886d45f197bfb51f9f4c335c46540dd43ab5: Status 404 returned error can't find the container with id b9915ee470f8cb6cf67614a28a28886d45f197bfb51f9f4c335c46540dd43ab5
Nov 04 02:24:09.437714 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.437684    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:24:09.437805 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.437747    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:24:09.437805 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.437773    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:24:09.518140 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.518113    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bccedafc-3320-4d2c-9a1f-a38dd01adac2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bccedafc-3320-4d2c-9a1f-a38dd01adac2-maas-api-tls\") pod \"bccedafc-3320-4d2c-9a1f-a38dd01adac2\" (UID: \"bccedafc-3320-4d2c-9a1f-a38dd01adac2\") "
Nov 04 02:24:09.518268 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.518157    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bccedafc-3320-4d2c-9a1f-a38dd01adac2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/bccedafc-3320-4d2c-9a1f-a38dd01adac2-metrics-certs\") pod \"bccedafc-3320-4d2c-9a1f-a38dd01adac2\" (UID: \"bccedafc-3320-4d2c-9a1f-a38dd01adac2\") "
Nov 04 02:24:09.518268 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.518222    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bccedafc-3320-4d2c-9a1f-a38dd01adac2-kube-api-access-cfqrp\" (UniqueName: \"kubernetes.io/projected/bccedafc-3320-4d2c-9a1f-a38dd01adac2-kube-api-access-cfqrp\") pod \"bccedafc-3320-4d2c-9a1f-a38dd01adac2\" (UID: \"bccedafc-3320-4d2c-9a1f-a38dd01adac2\") "
Nov 04 02:24:09.518401 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.518328    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bccedafc-3320-4d2c-9a1f-a38dd01adac2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bccedafc-3320-4d2c-9a1f-a38dd01adac2-openshift-service-ca\") pod \"bccedafc-3320-4d2c-9a1f-a38dd01adac2\" (UID: \"bccedafc-3320-4d2c-9a1f-a38dd01adac2\") "
Nov 04 02:24:09.518734 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.518711    4969 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62b73809-c52f-4c4a-915d-f03e77df2068-plugins-config-volume\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:09.518875 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.518736    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7nmf\" (UniqueName: \"kubernetes.io/projected/6e4b23e2-526a-464c-bb05-2f156da5652b-kube-api-access-w7nmf\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:09.518875 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.518748    4969 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e4b23e2-526a-464c-bb05-2f156da5652b-plugins-config-volume\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:09.518875 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.518764    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8ct6\" (UniqueName: \"kubernetes.io/projected/62b73809-c52f-4c4a-915d-f03e77df2068-kube-api-access-r8ct6\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:09.519218 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.519196    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bccedafc-3320-4d2c-9a1f-a38dd01adac2-openshift-service-ca" pod "bccedafc-3320-4d2c-9a1f-a38dd01adac2" (UID: "bccedafc-3320-4d2c-9a1f-a38dd01adac2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:24:09.520603 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.520576    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bccedafc-3320-4d2c-9a1f-a38dd01adac2-maas-api-tls" pod "bccedafc-3320-4d2c-9a1f-a38dd01adac2" (UID: "bccedafc-3320-4d2c-9a1f-a38dd01adac2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:24:09.521137 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.521086    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bccedafc-3320-4d2c-9a1f-a38dd01adac2-metrics-certs" pod "bccedafc-3320-4d2c-9a1f-a38dd01adac2" (UID: "bccedafc-3320-4d2c-9a1f-a38dd01adac2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:24:09.521483 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.521459    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bccedafc-3320-4d2c-9a1f-a38dd01adac2-kube-api-access-cfqrp" pod "bccedafc-3320-4d2c-9a1f-a38dd01adac2" (UID: "bccedafc-3320-4d2c-9a1f-a38dd01adac2"). InnerVolumeSpecName "kube-api-access-cfqrp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:24:09.620027 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.619985    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfqrp\" (UniqueName: \"kubernetes.io/projected/bccedafc-3320-4d2c-9a1f-a38dd01adac2-kube-api-access-cfqrp\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:09.620027 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.620024    4969 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bccedafc-3320-4d2c-9a1f-a38dd01adac2-openshift-service-ca\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:09.620275 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.620041    4969 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bccedafc-3320-4d2c-9a1f-a38dd01adac2-maas-api-tls\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:09.620275 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.620056    4969 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bccedafc-3320-4d2c-9a1f-a38dd01adac2-metrics-certs\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:09.802734 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.802657    4969 generic.go:356] "Generic (PLEG): container finished" podID="62b73809-c52f-4c4a-915d-f03e77df2068" containerID="5dd339c514d916178d34e74a1e92edf5ca7dec1d38b16275bbb7199b196a27eb" exitCode=0
Nov 04 02:24:09.802734 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.802722    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk"
Nov 04 02:24:09.803391 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.802739    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk" event={"ID":"62b73809-c52f-4c4a-915d-f03e77df2068","Type":"ContainerDied","Data":"5dd339c514d916178d34e74a1e92edf5ca7dec1d38b16275bbb7199b196a27eb"}
Nov 04 02:24:09.803391 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.802798    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk" event={"ID":"62b73809-c52f-4c4a-915d-f03e77df2068","Type":"ContainerDied","Data":"82503543b0a22aa3fea3cb563c50afda8932ac65df6ec8ed99bf04863f4fad3d"}
Nov 04 02:24:09.803391 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.802833    4969 scope.go:122] "RemoveContainer" containerID="5dd339c514d916178d34e74a1e92edf5ca7dec1d38b16275bbb7199b196a27eb"
Nov 04 02:24:09.804691 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.804670    4969 generic.go:356] "Generic (PLEG): container finished" podID="bccedafc-3320-4d2c-9a1f-a38dd01adac2" containerID="446063e0152795fdfbbb12d23281e5f54a35598eabb6ea2007e0ad14d90c51ca" exitCode=0
Nov 04 02:24:09.804814 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.804801    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv"
Nov 04 02:24:09.805236 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.805214    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv" event={"ID":"bccedafc-3320-4d2c-9a1f-a38dd01adac2","Type":"ContainerDied","Data":"446063e0152795fdfbbb12d23281e5f54a35598eabb6ea2007e0ad14d90c51ca"}
Nov 04 02:24:09.805319 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.805261    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv" event={"ID":"bccedafc-3320-4d2c-9a1f-a38dd01adac2","Type":"ContainerDied","Data":"bec237e1eb8d8d20558b9ab608cf3a2ff75afda01d1d14cb0936ebb24c46f313"}
Nov 04 02:24:09.807383 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.807356    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b" event={"ID":"0ba3e718-6552-472a-8002-259a7ffc4b89","Type":"ContainerStarted","Data":"233d19d0f95f5760fb2c14d07223859bad953f8ad000d1a8b50bc9bf9ce98434"}
Nov 04 02:24:09.807484 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.807403    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b" event={"ID":"0ba3e718-6552-472a-8002-259a7ffc4b89","Type":"ContainerStarted","Data":"b9915ee470f8cb6cf67614a28a28886d45f197bfb51f9f4c335c46540dd43ab5"}
Nov 04 02:24:09.809178 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.809149    4969 generic.go:356] "Generic (PLEG): container finished" podID="6e4b23e2-526a-464c-bb05-2f156da5652b" containerID="0a4f86cf1106335275af3546e544778829c13969a52c56675e077dd5af200500" exitCode=0
Nov 04 02:24:09.809277 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.809206    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt"
Nov 04 02:24:09.809277 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.809263    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt" event={"ID":"6e4b23e2-526a-464c-bb05-2f156da5652b","Type":"ContainerDied","Data":"0a4f86cf1106335275af3546e544778829c13969a52c56675e077dd5af200500"}
Nov 04 02:24:09.809393 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.809300    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt" event={"ID":"6e4b23e2-526a-464c-bb05-2f156da5652b","Type":"ContainerDied","Data":"9da866380901805a991f6cd741a4ed8b94f2861b68385bc2161d2259e18b60e6"}
Nov 04 02:24:09.823750 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.822739    4969 scope.go:122] "RemoveContainer" containerID="5dd339c514d916178d34e74a1e92edf5ca7dec1d38b16275bbb7199b196a27eb"
Nov 04 02:24:09.823750 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:24:09.823608    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5dd339c514d916178d34e74a1e92edf5ca7dec1d38b16275bbb7199b196a27eb\": container with ID starting with 5dd339c514d916178d34e74a1e92edf5ca7dec1d38b16275bbb7199b196a27eb not found: ID does not exist" containerID="5dd339c514d916178d34e74a1e92edf5ca7dec1d38b16275bbb7199b196a27eb"
Nov 04 02:24:09.823750 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.823646    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dd339c514d916178d34e74a1e92edf5ca7dec1d38b16275bbb7199b196a27eb"} err="failed to get container status \"5dd339c514d916178d34e74a1e92edf5ca7dec1d38b16275bbb7199b196a27eb\": rpc error: code = NotFound desc = could not find container \"5dd339c514d916178d34e74a1e92edf5ca7dec1d38b16275bbb7199b196a27eb\": container with ID starting with 5dd339c514d916178d34e74a1e92edf5ca7dec1d38b16275bbb7199b196a27eb not found: ID does not exist"
Nov 04 02:24:09.823750 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.823669    4969 scope.go:122] "RemoveContainer" containerID="446063e0152795fdfbbb12d23281e5f54a35598eabb6ea2007e0ad14d90c51ca"
Nov 04 02:24:09.837537 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.837512    4969 scope.go:122] "RemoveContainer" containerID="446063e0152795fdfbbb12d23281e5f54a35598eabb6ea2007e0ad14d90c51ca"
Nov 04 02:24:09.837837 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:24:09.837817    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"446063e0152795fdfbbb12d23281e5f54a35598eabb6ea2007e0ad14d90c51ca\": container with ID starting with 446063e0152795fdfbbb12d23281e5f54a35598eabb6ea2007e0ad14d90c51ca not found: ID does not exist" containerID="446063e0152795fdfbbb12d23281e5f54a35598eabb6ea2007e0ad14d90c51ca"
Nov 04 02:24:09.837910 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.837844    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"446063e0152795fdfbbb12d23281e5f54a35598eabb6ea2007e0ad14d90c51ca"} err="failed to get container status \"446063e0152795fdfbbb12d23281e5f54a35598eabb6ea2007e0ad14d90c51ca\": rpc error: code = NotFound desc = could not find container \"446063e0152795fdfbbb12d23281e5f54a35598eabb6ea2007e0ad14d90c51ca\": container with ID starting with 446063e0152795fdfbbb12d23281e5f54a35598eabb6ea2007e0ad14d90c51ca not found: ID does not exist"
Nov 04 02:24:09.837910 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.837862    4969 scope.go:122] "RemoveContainer" containerID="0a4f86cf1106335275af3546e544778829c13969a52c56675e077dd5af200500"
Nov 04 02:24:09.849888 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.849872    4969 scope.go:122] "RemoveContainer" containerID="0a4f86cf1106335275af3546e544778829c13969a52c56675e077dd5af200500"
Nov 04 02:24:09.850269 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:24:09.850246    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a4f86cf1106335275af3546e544778829c13969a52c56675e077dd5af200500\": container with ID starting with 0a4f86cf1106335275af3546e544778829c13969a52c56675e077dd5af200500 not found: ID does not exist" containerID="0a4f86cf1106335275af3546e544778829c13969a52c56675e077dd5af200500"
Nov 04 02:24:09.850363 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.850277    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a4f86cf1106335275af3546e544778829c13969a52c56675e077dd5af200500"} err="failed to get container status \"0a4f86cf1106335275af3546e544778829c13969a52c56675e077dd5af200500\": rpc error: code = NotFound desc = could not find container \"0a4f86cf1106335275af3546e544778829c13969a52c56675e077dd5af200500\": container with ID starting with 0a4f86cf1106335275af3546e544778829c13969a52c56675e077dd5af200500 not found: ID does not exist"
Nov 04 02:24:09.992782 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:09.992480    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt"]
Nov 04 02:24:10.013133 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:10.007759    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-dd5d18-fb99975d4-hj5bt"]
Nov 04 02:24:10.013133 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:10.012897    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:10.014190 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:10.014164    4969 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 04 02:24:10.014320 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:10.014224    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b" podUID="0ba3e718-6552-472a-8002-259a7ffc4b89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 04 02:24:10.038662 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:10.038629    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:10.095415 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:10.095332    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk"]
Nov 04 02:24:10.099346 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:10.099319    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-dd5d18-8588df9975-6nkpk"]
Nov 04 02:24:10.205060 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:10.205024    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="62b73809-c52f-4c4a-915d-f03e77df2068" path="/var/lib/kubelet/pods/62b73809-c52f-4c4a-915d-f03e77df2068/volumes"
Nov 04 02:24:10.205661 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:10.205636    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6e4b23e2-526a-464c-bb05-2f156da5652b" path="/var/lib/kubelet/pods/6e4b23e2-526a-464c-bb05-2f156da5652b/volumes"
Nov 04 02:24:10.222654 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:10.222600    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b" podStartSLOduration=2.222583537 podStartE2EDuration="2.222583537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:24:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:24:10.21823826 +0000 UTC m=+1086.144460893" watchObservedRunningTime="2026-11-04 02:24:10.222583537 +0000 UTC m=+1086.148806171"
Nov 04 02:24:10.227923 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:10.227900    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:10.339841 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:10.339805    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv"]
Nov 04 02:24:10.343373 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:10.343345    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-dd5d18-7b56dff9bb-gj4rv"]
Nov 04 02:24:11.001939 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:11.001905    4969 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 04 02:24:11.002445 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:11.001969    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b" podUID="0ba3e718-6552-472a-8002-259a7ffc4b89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 04 02:24:11.992577 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:11.992536    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"]
Nov 04 02:24:11.993162 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:11.993145    4969 cpu_manager.go:414] "containerMap: removing container" podUID="bccedafc-3320-4d2c-9a1f-a38dd01adac2" containerName="maas-api"
Nov 04 02:24:11.993222 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:11.993162    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bccedafc-3320-4d2c-9a1f-a38dd01adac2" containerName="maas-api"
Nov 04 02:24:11.993222 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:11.993187    4969 cpu_manager.go:414] "containerMap: removing container" podUID="6e4b23e2-526a-464c-bb05-2f156da5652b" containerName="payload-processing"
Nov 04 02:24:11.993222 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:11.993192    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6e4b23e2-526a-464c-bb05-2f156da5652b" containerName="payload-processing"
Nov 04 02:24:11.993222 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:11.993201    4969 cpu_manager.go:414] "containerMap: removing container" podUID="62b73809-c52f-4c4a-915d-f03e77df2068" containerName="payload-pre-processing"
Nov 04 02:24:11.993222 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:11.993207    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="62b73809-c52f-4c4a-915d-f03e77df2068" containerName="payload-pre-processing"
Nov 04 02:24:11.993375 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:11.993285    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="bccedafc-3320-4d2c-9a1f-a38dd01adac2" containerName="maas-api"
Nov 04 02:24:11.993375 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:11.993296    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="6e4b23e2-526a-464c-bb05-2f156da5652b" containerName="payload-processing"
Nov 04 02:24:11.993375 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:11.993302    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="62b73809-c52f-4c4a-915d-f03e77df2068" containerName="payload-pre-processing"
Nov 04 02:24:11.998429 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:11.998402    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.001862 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.001827    4969 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 04 02:24:12.002007 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.001885    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b" podUID="0ba3e718-6552-472a-8002-259a7ffc4b89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 04 02:24:12.050630 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.050605    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-9adedd-openshift-default-dockercfg-4x2k2\""
Nov 04 02:24:12.063116 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.059637    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istio-envoy\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.063116 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.059738    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-credential-socket\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.063116 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.059768    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istio-podinfo\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.063116 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.059817    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istio-data\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.063116 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.059848    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istiod-ca-cert\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.063116 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.059879    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-workload-socket\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.063116 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.059904    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4bvz\" (UniqueName: \"kubernetes.io/projected/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-kube-api-access-l4bvz\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.063116 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.059944    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-workload-certs\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.063116 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.060010    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istio-token\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.104945 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.104911    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"]
Nov 04 02:24:12.163666 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.163628    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-credential-socket\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.167393 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.167366    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istio-podinfo\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.168003 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.167983    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istio-data\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.168167 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.168147    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istiod-ca-cert\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.168596 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.168575    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-workload-socket\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.168720 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.167300    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-credential-socket\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.177111 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.169044    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istio-data\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.177409 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.177379    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-workload-socket\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.177602 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.177582    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4bvz\" (UniqueName: \"kubernetes.io/projected/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-kube-api-access-l4bvz\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.183197 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.183173    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-workload-certs\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.183703 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.183686    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istio-token\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.184174 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.184152    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istio-envoy\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.184403 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.177860    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istiod-ca-cert\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.184825 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.183588    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-workload-certs\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.187636 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.187616    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istio-envoy\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.187882 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.187861    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istio-podinfo\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.202891 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.202566    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bccedafc-3320-4d2c-9a1f-a38dd01adac2" path="/var/lib/kubelet/pods/bccedafc-3320-4d2c-9a1f-a38dd01adac2/volumes"
Nov 04 02:24:12.241208 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.241182    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4bvz\" (UniqueName: \"kubernetes.io/projected/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-kube-api-access-l4bvz\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.250292 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.250242    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istio-token\") pod \"e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") " pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.320231 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.318836    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:12.794149 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.793535    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"]
Nov 04 02:24:12.812685 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:24:12.812658    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9f1b6e2_7dd2_41ac_8c57_f35f0c2a4eec.slice/crio-7b4dd960b8a3fc47305ac87e33f1a64e79d4bd7f9e02b83f8a913bb03a5d8209 WatchSource:0}: Error finding container 7b4dd960b8a3fc47305ac87e33f1a64e79d4bd7f9e02b83f8a913bb03a5d8209: Status 404 returned error can't find the container with id 7b4dd960b8a3fc47305ac87e33f1a64e79d4bd7f9e02b83f8a913bb03a5d8209
Nov 04 02:24:12.815365 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.815338    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:24:12.815434 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.815414    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:24:12.815475 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.815443    4969 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Nov 04 02:24:12.838483 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:12.838455    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz" event={"ID":"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec","Type":"ContainerStarted","Data":"7b4dd960b8a3fc47305ac87e33f1a64e79d4bd7f9e02b83f8a913bb03a5d8209"}
Nov 04 02:24:13.004115 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:13.003435    4969 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 04 02:24:13.004115 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:13.003496    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b" podUID="0ba3e718-6552-472a-8002-259a7ffc4b89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 04 02:24:13.633674 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:13.633645    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:13.816123 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:13.812361    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:13.867116 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:13.866053    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz" event={"ID":"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec","Type":"ContainerStarted","Data":"5c3bb2291e15376dcee531e4abc8b41278a16c2382562bd95eaae6a0340fc0d4"}
Nov 04 02:24:14.004174 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:14.002477    4969 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 04 02:24:14.004174 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:14.002542    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b" podUID="0ba3e718-6552-472a-8002-259a7ffc4b89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 04 02:24:14.030117 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:14.029582    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:14.104117 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:14.103765    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz" podStartSLOduration=3.103744025 podStartE2EDuration="3.103744025s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:24:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:24:14.03763986 +0000 UTC m=+1089.963862495" watchObservedRunningTime="2026-11-04 02:24:14.103744025 +0000 UTC m=+1090.029966661"
Nov 04 02:24:14.322268 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:14.322234    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:14.322605 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:14.322584    4969 patch_prober.go:29] interesting pod/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 04 02:24:14.322759 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:14.322735    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz" podUID="c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 04 02:24:15.010156 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:15.002475    4969 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 04 02:24:15.010156 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:15.002541    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b" podUID="0ba3e718-6552-472a-8002-259a7ffc4b89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 04 02:24:15.327175 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:15.322492    4969 patch_prober.go:29] interesting pod/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 04 02:24:15.327175 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:15.322560    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz" podUID="c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 04 02:24:15.473479 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:15.473429    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:15.711004 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:15.710930    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:15.986172 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:15.985182    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"]
Nov 04 02:24:16.003266 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:16.003219    4969 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Nov 04 02:24:16.003525 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:16.003501    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b" podUID="0ba3e718-6552-472a-8002-259a7ffc4b89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Nov 04 02:24:16.014207 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:16.014180    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"]
Nov 04 02:24:16.014893 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:16.014867    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b" podUID="0ba3e718-6552-472a-8002-259a7ffc4b89" containerName="istio-proxy" containerID="cri-o://233d19d0f95f5760fb2c14d07223859bad953f8ad000d1a8b50bc9bf9ce98434" gracePeriod=30
Nov 04 02:24:16.015265 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:16.015243    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns" podUID="4e50fee5-a7b8-436b-8685-ff90f478d369" containerName="istio-proxy" containerID="cri-o://a05a0b752591a5f60a65d5e38b85a4a29aefb23e1fcc29ee994d61c91735e5a3" gracePeriod=30
Nov 04 02:24:16.019417 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:16.019275    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:16.320558 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:16.320519    4969 patch_prober.go:29] interesting pod/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 04 02:24:16.320758 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:16.320590    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz" podUID="c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 04 02:24:16.832025 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:16.831988    4969 patch_prober.go:29] interesting pod/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 02:24:16.832247 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:16.832052    4969 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns" podUID="4e50fee5-a7b8-436b-8685-ff90f478d369" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 02:24:17.320438 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:17.320398    4969 patch_prober.go:29] interesting pod/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 04 02:24:17.320986 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:17.320452    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz" podUID="c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 04 02:24:18.320240 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:18.320209    4969 patch_prober.go:29] interesting pod/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 04 02:24:18.320410 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:18.320260    4969 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz" podUID="c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 04 02:24:19.412716 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:19.412683    4969 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:19.413280 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:19.413260    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:19.414146 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:19.414123    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:20.036544 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:20.036513    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:20.220297 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:20.220261    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:21.039495 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.039469    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:21.277147 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.277124    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:24:21.439697 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.439606    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4e50fee5-a7b8-436b-8685-ff90f478d369-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e50fee5-a7b8-436b-8685-ff90f478d369-istio-podinfo\") pod \"4e50fee5-a7b8-436b-8685-ff90f478d369\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") "
Nov 04 02:24:21.439697 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.439674    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-istio-data\") pod \"4e50fee5-a7b8-436b-8685-ff90f478d369\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") "
Nov 04 02:24:21.439922 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.439722    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4e50fee5-a7b8-436b-8685-ff90f478d369-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e50fee5-a7b8-436b-8685-ff90f478d369-istiod-ca-cert\") pod \"4e50fee5-a7b8-436b-8685-ff90f478d369\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") "
Nov 04 02:24:21.439922 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.439740    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e50fee5-a7b8-436b-8685-ff90f478d369-kube-api-access-cvgc6\" (UniqueName: \"kubernetes.io/projected/4e50fee5-a7b8-436b-8685-ff90f478d369-kube-api-access-cvgc6\") pod \"4e50fee5-a7b8-436b-8685-ff90f478d369\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") "
Nov 04 02:24:21.439922 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.439785    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-credential-socket\") pod \"4e50fee5-a7b8-436b-8685-ff90f478d369\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") "
Nov 04 02:24:21.439922 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.439821    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-workload-certs\") pod \"4e50fee5-a7b8-436b-8685-ff90f478d369\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") "
Nov 04 02:24:21.439922 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.439883    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-istio-envoy\") pod \"4e50fee5-a7b8-436b-8685-ff90f478d369\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") "
Nov 04 02:24:21.439922 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.439912    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e50fee5-a7b8-436b-8685-ff90f478d369-istio-token\" (UniqueName: \"kubernetes.io/projected/4e50fee5-a7b8-436b-8685-ff90f478d369-istio-token\") pod \"4e50fee5-a7b8-436b-8685-ff90f478d369\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") "
Nov 04 02:24:21.440267 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.439983    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-workload-socket\") pod \"4e50fee5-a7b8-436b-8685-ff90f478d369\" (UID: \"4e50fee5-a7b8-436b-8685-ff90f478d369\") "
Nov 04 02:24:21.440267 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.440054    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-istio-data" pod "4e50fee5-a7b8-436b-8685-ff90f478d369" (UID: "4e50fee5-a7b8-436b-8685-ff90f478d369"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:24:21.440267 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.440156    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-credential-socket" pod "4e50fee5-a7b8-436b-8685-ff90f478d369" (UID: "4e50fee5-a7b8-436b-8685-ff90f478d369"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:24:21.440267 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.440169    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e50fee5-a7b8-436b-8685-ff90f478d369-istiod-ca-cert" pod "4e50fee5-a7b8-436b-8685-ff90f478d369" (UID: "4e50fee5-a7b8-436b-8685-ff90f478d369"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:24:21.440473 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.440365    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-workload-socket" pod "4e50fee5-a7b8-436b-8685-ff90f478d369" (UID: "4e50fee5-a7b8-436b-8685-ff90f478d369"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:24:21.440473 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.440433    4969 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-istio-data\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:21.440473 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.440455    4969 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e50fee5-a7b8-436b-8685-ff90f478d369-istiod-ca-cert\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:21.440686 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.440473    4969 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-credential-socket\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:21.440686 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.440578    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-workload-certs" pod "4e50fee5-a7b8-436b-8685-ff90f478d369" (UID: "4e50fee5-a7b8-436b-8685-ff90f478d369"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:24:21.442200 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.442172    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-istio-envoy" pod "4e50fee5-a7b8-436b-8685-ff90f478d369" (UID: "4e50fee5-a7b8-436b-8685-ff90f478d369"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:24:21.442491 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.442471    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4e50fee5-a7b8-436b-8685-ff90f478d369-istio-podinfo" pod "4e50fee5-a7b8-436b-8685-ff90f478d369" (UID: "4e50fee5-a7b8-436b-8685-ff90f478d369"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:24:21.442668 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.442639    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e50fee5-a7b8-436b-8685-ff90f478d369-istio-token" pod "4e50fee5-a7b8-436b-8685-ff90f478d369" (UID: "4e50fee5-a7b8-436b-8685-ff90f478d369"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:24:21.442901 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.442880    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e50fee5-a7b8-436b-8685-ff90f478d369-kube-api-access-cvgc6" pod "4e50fee5-a7b8-436b-8685-ff90f478d369" (UID: "4e50fee5-a7b8-436b-8685-ff90f478d369"). InnerVolumeSpecName "kube-api-access-cvgc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:24:21.541736 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.541706    4969 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-workload-certs\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:21.541736 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.541732    4969 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-istio-envoy\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:21.541736 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.541741    4969 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e50fee5-a7b8-436b-8685-ff90f478d369-istio-token\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:21.541942 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.541750    4969 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e50fee5-a7b8-436b-8685-ff90f478d369-workload-socket\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:21.541942 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.541758    4969 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e50fee5-a7b8-436b-8685-ff90f478d369-istio-podinfo\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:21.541942 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.541767    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvgc6\" (UniqueName: \"kubernetes.io/projected/4e50fee5-a7b8-436b-8685-ff90f478d369-kube-api-access-cvgc6\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:21.961682 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.961647    4969 generic.go:356] "Generic (PLEG): container finished" podID="4e50fee5-a7b8-436b-8685-ff90f478d369" containerID="a05a0b752591a5f60a65d5e38b85a4a29aefb23e1fcc29ee994d61c91735e5a3" exitCode=0
Nov 04 02:24:21.961855 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.961695    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"
Nov 04 02:24:21.961855 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.961728    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns" event={"ID":"4e50fee5-a7b8-436b-8685-ff90f478d369","Type":"ContainerDied","Data":"a05a0b752591a5f60a65d5e38b85a4a29aefb23e1fcc29ee994d61c91735e5a3"}
Nov 04 02:24:21.961855 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.961784    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns" event={"ID":"4e50fee5-a7b8-436b-8685-ff90f478d369","Type":"ContainerDied","Data":"92ec2fcfa8b6ad8b432df874eb09728524e691957d0a420b5b45f83d7ef92cf4"}
Nov 04 02:24:21.961855 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.961813    4969 scope.go:122] "RemoveContainer" containerID="a05a0b752591a5f60a65d5e38b85a4a29aefb23e1fcc29ee994d61c91735e5a3"
Nov 04 02:24:21.971789 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.971765    4969 scope.go:122] "RemoveContainer" containerID="a05a0b752591a5f60a65d5e38b85a4a29aefb23e1fcc29ee994d61c91735e5a3"
Nov 04 02:24:21.972082 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:24:21.972063    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a05a0b752591a5f60a65d5e38b85a4a29aefb23e1fcc29ee994d61c91735e5a3\": container with ID starting with a05a0b752591a5f60a65d5e38b85a4a29aefb23e1fcc29ee994d61c91735e5a3 not found: ID does not exist" containerID="a05a0b752591a5f60a65d5e38b85a4a29aefb23e1fcc29ee994d61c91735e5a3"
Nov 04 02:24:21.972198 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:21.972104    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a05a0b752591a5f60a65d5e38b85a4a29aefb23e1fcc29ee994d61c91735e5a3"} err="failed to get container status \"a05a0b752591a5f60a65d5e38b85a4a29aefb23e1fcc29ee994d61c91735e5a3\": rpc error: code = NotFound desc = could not find container \"a05a0b752591a5f60a65d5e38b85a4a29aefb23e1fcc29ee994d61c91735e5a3\": container with ID starting with a05a0b752591a5f60a65d5e38b85a4a29aefb23e1fcc29ee994d61c91735e5a3 not found: ID does not exist"
Nov 04 02:24:22.128810 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.128772    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"]
Nov 04 02:24:22.177951 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.177917    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-ds6ns"]
Nov 04 02:24:22.201804 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.201780    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4e50fee5-a7b8-436b-8685-ff90f478d369" path="/var/lib/kubelet/pods/4e50fee5-a7b8-436b-8685-ff90f478d369/volumes"
Nov 04 02:24:22.589258 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.589231    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:22.751270 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.751195    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0ba3e718-6552-472a-8002-259a7ffc4b89-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ba3e718-6552-472a-8002-259a7ffc4b89-istiod-ca-cert\") pod \"0ba3e718-6552-472a-8002-259a7ffc4b89\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") "
Nov 04 02:24:22.751436 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.751286    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-istio-data\") pod \"0ba3e718-6552-472a-8002-259a7ffc4b89\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") "
Nov 04 02:24:22.751436 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.751321    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-workload-certs\") pod \"0ba3e718-6552-472a-8002-259a7ffc4b89\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") "
Nov 04 02:24:22.751436 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.751356    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ba3e718-6552-472a-8002-259a7ffc4b89-kube-api-access-tfm2j\" (UniqueName: \"kubernetes.io/projected/0ba3e718-6552-472a-8002-259a7ffc4b89-kube-api-access-tfm2j\") pod \"0ba3e718-6552-472a-8002-259a7ffc4b89\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") "
Nov 04 02:24:22.751436 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.751380    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-credential-socket\") pod \"0ba3e718-6552-472a-8002-259a7ffc4b89\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") "
Nov 04 02:24:22.751436 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.751421    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-istio-envoy\") pod \"0ba3e718-6552-472a-8002-259a7ffc4b89\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") "
Nov 04 02:24:22.751717 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.751450    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-workload-socket\") pod \"0ba3e718-6552-472a-8002-259a7ffc4b89\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") "
Nov 04 02:24:22.751717 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.751493    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ba3e718-6552-472a-8002-259a7ffc4b89-istio-token\" (UniqueName: \"kubernetes.io/projected/0ba3e718-6552-472a-8002-259a7ffc4b89-istio-token\") pod \"0ba3e718-6552-472a-8002-259a7ffc4b89\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") "
Nov 04 02:24:22.751717 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.751524    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0ba3e718-6552-472a-8002-259a7ffc4b89-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ba3e718-6552-472a-8002-259a7ffc4b89-istio-podinfo\") pod \"0ba3e718-6552-472a-8002-259a7ffc4b89\" (UID: \"0ba3e718-6552-472a-8002-259a7ffc4b89\") "
Nov 04 02:24:22.751717 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.751649    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-istio-data" pod "0ba3e718-6552-472a-8002-259a7ffc4b89" (UID: "0ba3e718-6552-472a-8002-259a7ffc4b89"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:24:22.751717 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.751666    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-workload-certs" pod "0ba3e718-6552-472a-8002-259a7ffc4b89" (UID: "0ba3e718-6552-472a-8002-259a7ffc4b89"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:24:22.751717 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.751685    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ba3e718-6552-472a-8002-259a7ffc4b89-istiod-ca-cert" pod "0ba3e718-6552-472a-8002-259a7ffc4b89" (UID: "0ba3e718-6552-472a-8002-259a7ffc4b89"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:24:22.751996 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.751808    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-credential-socket" pod "0ba3e718-6552-472a-8002-259a7ffc4b89" (UID: "0ba3e718-6552-472a-8002-259a7ffc4b89"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:24:22.751996 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.751926    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-workload-socket" pod "0ba3e718-6552-472a-8002-259a7ffc4b89" (UID: "0ba3e718-6552-472a-8002-259a7ffc4b89"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:24:22.752274 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.752238    4969 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ba3e718-6552-472a-8002-259a7ffc4b89-istiod-ca-cert\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:22.752274 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.752265    4969 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-istio-data\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:22.752464 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.752278    4969 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-workload-certs\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:22.752464 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.752296    4969 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-credential-socket\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:22.752464 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.752323    4969 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-workload-socket\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:22.753887 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.753862    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ba3e718-6552-472a-8002-259a7ffc4b89-kube-api-access-tfm2j" pod "0ba3e718-6552-472a-8002-259a7ffc4b89" (UID: "0ba3e718-6552-472a-8002-259a7ffc4b89"). InnerVolumeSpecName "kube-api-access-tfm2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:24:22.754024 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.753930    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-istio-envoy" pod "0ba3e718-6552-472a-8002-259a7ffc4b89" (UID: "0ba3e718-6552-472a-8002-259a7ffc4b89"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:24:22.754083 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.754028    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0ba3e718-6552-472a-8002-259a7ffc4b89-istio-podinfo" pod "0ba3e718-6552-472a-8002-259a7ffc4b89" (UID: "0ba3e718-6552-472a-8002-259a7ffc4b89"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:24:22.754083 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.754021    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ba3e718-6552-472a-8002-259a7ffc4b89-istio-token" pod "0ba3e718-6552-472a-8002-259a7ffc4b89" (UID: "0ba3e718-6552-472a-8002-259a7ffc4b89"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:24:22.852948 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.852924    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfm2j\" (UniqueName: \"kubernetes.io/projected/0ba3e718-6552-472a-8002-259a7ffc4b89-kube-api-access-tfm2j\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:22.852948 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.852949    4969 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ba3e718-6552-472a-8002-259a7ffc4b89-istio-envoy\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:22.853084 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.852958    4969 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ba3e718-6552-472a-8002-259a7ffc4b89-istio-token\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:22.853084 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.852967    4969 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ba3e718-6552-472a-8002-259a7ffc4b89-istio-podinfo\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:22.968894 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.968861    4969 generic.go:356] "Generic (PLEG): container finished" podID="0ba3e718-6552-472a-8002-259a7ffc4b89" containerID="233d19d0f95f5760fb2c14d07223859bad953f8ad000d1a8b50bc9bf9ce98434" exitCode=0
Nov 04 02:24:22.969018 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.968909    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"
Nov 04 02:24:22.969018 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.968946    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b" event={"ID":"0ba3e718-6552-472a-8002-259a7ffc4b89","Type":"ContainerDied","Data":"233d19d0f95f5760fb2c14d07223859bad953f8ad000d1a8b50bc9bf9ce98434"}
Nov 04 02:24:22.969018 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.968999    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b" event={"ID":"0ba3e718-6552-472a-8002-259a7ffc4b89","Type":"ContainerDied","Data":"b9915ee470f8cb6cf67614a28a28886d45f197bfb51f9f4c335c46540dd43ab5"}
Nov 04 02:24:22.969185 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.969030    4969 scope.go:122] "RemoveContainer" containerID="233d19d0f95f5760fb2c14d07223859bad953f8ad000d1a8b50bc9bf9ce98434"
Nov 04 02:24:22.978608 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.978592    4969 scope.go:122] "RemoveContainer" containerID="233d19d0f95f5760fb2c14d07223859bad953f8ad000d1a8b50bc9bf9ce98434"
Nov 04 02:24:22.978865 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:24:22.978850    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"233d19d0f95f5760fb2c14d07223859bad953f8ad000d1a8b50bc9bf9ce98434\": container with ID starting with 233d19d0f95f5760fb2c14d07223859bad953f8ad000d1a8b50bc9bf9ce98434 not found: ID does not exist" containerID="233d19d0f95f5760fb2c14d07223859bad953f8ad000d1a8b50bc9bf9ce98434"
Nov 04 02:24:22.978913 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:22.978871    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"233d19d0f95f5760fb2c14d07223859bad953f8ad000d1a8b50bc9bf9ce98434"} err="failed to get container status \"233d19d0f95f5760fb2c14d07223859bad953f8ad000d1a8b50bc9bf9ce98434\": rpc error: code = NotFound desc = could not find container \"233d19d0f95f5760fb2c14d07223859bad953f8ad000d1a8b50bc9bf9ce98434\": container with ID starting with 233d19d0f95f5760fb2c14d07223859bad953f8ad000d1a8b50bc9bf9ce98434 not found: ID does not exist"
Nov 04 02:24:23.136186 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:23.136154    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"]
Nov 04 02:24:23.186059 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:23.186028    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-dd5d18-openshift-default-c7c7d86c-f5r8b"]
Nov 04 02:24:24.201470 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:24.201436    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0ba3e718-6552-472a-8002-259a7ffc4b89" path="/var/lib/kubelet/pods/0ba3e718-6552-472a-8002-259a7ffc4b89/volumes"
Nov 04 02:24:25.284771 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:25.284736    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg"]
Nov 04 02:24:25.285171 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:25.284992    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg" podUID="ea0b94ed-20c6-4b3a-abd6-e39992324ec6" containerName="maas-api" containerID="cri-o://450b37f4b7b7782541cb836bd4ae4e4fd1e424f369f3127df0e7c7afbd1d4a84" gracePeriod=30
Nov 04 02:24:25.422757 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:25.422729    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:25.543064 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:25.543003    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg"
Nov 04 02:24:25.681028 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:25.680995    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-openshift-service-ca\") pod \"ea0b94ed-20c6-4b3a-abd6-e39992324ec6\" (UID: \"ea0b94ed-20c6-4b3a-abd6-e39992324ec6\") "
Nov 04 02:24:25.681242 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:25.681078    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-metrics-certs\") pod \"ea0b94ed-20c6-4b3a-abd6-e39992324ec6\" (UID: \"ea0b94ed-20c6-4b3a-abd6-e39992324ec6\") "
Nov 04 02:24:25.681242 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:25.681168    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-maas-api-tls\") pod \"ea0b94ed-20c6-4b3a-abd6-e39992324ec6\" (UID: \"ea0b94ed-20c6-4b3a-abd6-e39992324ec6\") "
Nov 04 02:24:25.681242 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:25.681201    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-kube-api-access-ffjlw\" (UniqueName: \"kubernetes.io/projected/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-kube-api-access-ffjlw\") pod \"ea0b94ed-20c6-4b3a-abd6-e39992324ec6\" (UID: \"ea0b94ed-20c6-4b3a-abd6-e39992324ec6\") "
Nov 04 02:24:25.681518 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:25.681488    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-openshift-service-ca" pod "ea0b94ed-20c6-4b3a-abd6-e39992324ec6" (UID: "ea0b94ed-20c6-4b3a-abd6-e39992324ec6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:24:25.683593 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:25.683558    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-metrics-certs" pod "ea0b94ed-20c6-4b3a-abd6-e39992324ec6" (UID: "ea0b94ed-20c6-4b3a-abd6-e39992324ec6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:24:25.683760 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:25.683737    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-maas-api-tls" pod "ea0b94ed-20c6-4b3a-abd6-e39992324ec6" (UID: "ea0b94ed-20c6-4b3a-abd6-e39992324ec6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 02:24:25.683870 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:25.683838    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-kube-api-access-ffjlw" pod "ea0b94ed-20c6-4b3a-abd6-e39992324ec6" (UID: "ea0b94ed-20c6-4b3a-abd6-e39992324ec6"). InnerVolumeSpecName "kube-api-access-ffjlw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:24:25.783146 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:25.783084    4969 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-metrics-certs\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:25.783146 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:25.783146    4969 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-maas-api-tls\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:25.783361 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:25.783162    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffjlw\" (UniqueName: \"kubernetes.io/projected/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-kube-api-access-ffjlw\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:25.783361 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:25.783180    4969 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea0b94ed-20c6-4b3a-abd6-e39992324ec6-openshift-service-ca\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:25.989829 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:25.989719    4969 generic.go:356] "Generic (PLEG): container finished" podID="ea0b94ed-20c6-4b3a-abd6-e39992324ec6" containerID="450b37f4b7b7782541cb836bd4ae4e4fd1e424f369f3127df0e7c7afbd1d4a84" exitCode=0
Nov 04 02:24:25.989829 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:25.989787    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg"
Nov 04 02:24:25.989829 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:25.989814    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg" event={"ID":"ea0b94ed-20c6-4b3a-abd6-e39992324ec6","Type":"ContainerDied","Data":"450b37f4b7b7782541cb836bd4ae4e4fd1e424f369f3127df0e7c7afbd1d4a84"}
Nov 04 02:24:25.990075 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:25.989869    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg" event={"ID":"ea0b94ed-20c6-4b3a-abd6-e39992324ec6","Type":"ContainerDied","Data":"0c2af3a102492280f0c5fc7909285503895dd4460f83c39a938a561b1f896a67"}
Nov 04 02:24:25.990075 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:25.989900    4969 scope.go:122] "RemoveContainer" containerID="450b37f4b7b7782541cb836bd4ae4e4fd1e424f369f3127df0e7c7afbd1d4a84"
Nov 04 02:24:25.999893 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:25.999872    4969 scope.go:122] "RemoveContainer" containerID="450b37f4b7b7782541cb836bd4ae4e4fd1e424f369f3127df0e7c7afbd1d4a84"
Nov 04 02:24:26.000208 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:24:26.000185    4969 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"450b37f4b7b7782541cb836bd4ae4e4fd1e424f369f3127df0e7c7afbd1d4a84\": container with ID starting with 450b37f4b7b7782541cb836bd4ae4e4fd1e424f369f3127df0e7c7afbd1d4a84 not found: ID does not exist" containerID="450b37f4b7b7782541cb836bd4ae4e4fd1e424f369f3127df0e7c7afbd1d4a84"
Nov 04 02:24:26.000296 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:26.000216    4969 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"450b37f4b7b7782541cb836bd4ae4e4fd1e424f369f3127df0e7c7afbd1d4a84"} err="failed to get container status \"450b37f4b7b7782541cb836bd4ae4e4fd1e424f369f3127df0e7c7afbd1d4a84\": rpc error: code = NotFound desc = could not find container \"450b37f4b7b7782541cb836bd4ae4e4fd1e424f369f3127df0e7c7afbd1d4a84\": container with ID starting with 450b37f4b7b7782541cb836bd4ae4e4fd1e424f369f3127df0e7c7afbd1d4a84 not found: ID does not exist"
Nov 04 02:24:26.155792 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:26.155766    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg"]
Nov 04 02:24:26.204643 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:26.204616    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-9adedd-75c68f8cf6-qdwsg"]
Nov 04 02:24:26.991279 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:26.991250    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:27.275867 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:27.275834    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:27.771862 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:27.771831    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"]
Nov 04 02:24:27.772039 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:27.772018    4969 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz" podUID="c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec" containerName="istio-proxy" containerID="cri-o://5c3bb2291e15376dcee531e4abc8b41278a16c2382562bd95eaae6a0340fc0d4" gracePeriod=30
Nov 04 02:24:27.798914 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:27.798890    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:28.201671 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:28.201602    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ea0b94ed-20c6-4b3a-abd6-e39992324ec6" path="/var/lib/kubelet/pods/ea0b94ed-20c6-4b3a-abd6-e39992324ec6/volumes"
Nov 04 02:24:29.036482 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:29.036453    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:29.209277 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:29.209249    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:24:29.696930 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:29.696897    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-58bcb58bd7-lpkwf_9627a478-94ef-41ca-b965-3982e66f502f/authorino/0.log"
Nov 04 02:24:33.031076 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.031037    4969 generic.go:356] "Generic (PLEG): container finished" podID="c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec" containerID="5c3bb2291e15376dcee531e4abc8b41278a16c2382562bd95eaae6a0340fc0d4" exitCode=0
Nov 04 02:24:33.031436 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.031124    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz" event={"ID":"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec","Type":"ContainerDied","Data":"5c3bb2291e15376dcee531e4abc8b41278a16c2382562bd95eaae6a0340fc0d4"}
Nov 04 02:24:33.031436 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.031194    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz" event={"ID":"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec","Type":"ContainerDied","Data":"7b4dd960b8a3fc47305ac87e33f1a64e79d4bd7f9e02b83f8a913bb03a5d8209"}
Nov 04 02:24:33.031436 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.031224    4969 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7b4dd960b8a3fc47305ac87e33f1a64e79d4bd7f9e02b83f8a913bb03a5d8209"
Nov 04 02:24:33.034677 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.034611    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:33.046314 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.046296    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-kube-api-access-l4bvz\" (UniqueName: \"kubernetes.io/projected/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-kube-api-access-l4bvz\") pod \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") "
Nov 04 02:24:33.046483 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.046463    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istio-token\" (UniqueName: \"kubernetes.io/projected/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istio-token\") pod \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") "
Nov 04 02:24:33.046592 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.046579    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-workload-socket\") pod \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") "
Nov 04 02:24:33.046654 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.046621    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-workload-certs\") pod \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") "
Nov 04 02:24:33.046710 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.046678    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istiod-ca-cert\") pod \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") "
Nov 04 02:24:33.046772 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.046722    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-credential-socket\") pod \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") "
Nov 04 02:24:33.046830 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.046768    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istio-data\") pod \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") "
Nov 04 02:24:33.046883 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.046852    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istio-podinfo\") pod \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") "
Nov 04 02:24:33.046937 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.046925    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istio-envoy\") pod \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\" (UID: \"c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec\") "
Nov 04 02:24:33.047007 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.046857    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-workload-socket" pod "c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec" (UID: "c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:24:33.047070 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.046898    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-workload-certs" pod "c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec" (UID: "c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:24:33.047133 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.047071    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istiod-ca-cert" pod "c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec" (UID: "c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:24:33.047272 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.047246    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istio-data" pod "c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec" (UID: "c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:24:33.047381 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.047337    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-credential-socket" pod "c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec" (UID: "c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:24:33.047381 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.047323    4969 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-workload-certs\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:33.047486 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.047453    4969 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istiod-ca-cert\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:33.047486 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.047474    4969 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-workload-socket\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:33.048459 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.048436    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-kube-api-access-l4bvz" pod "c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec" (UID: "c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec"). InnerVolumeSpecName "kube-api-access-l4bvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:24:33.048802 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.048786    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istio-podinfo" pod "c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec" (UID: "c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 02:24:33.048924 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.048906    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istio-token" pod "c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec" (UID: "c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:24:33.049184 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.049168    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istio-envoy" pod "c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec" (UID: "c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:24:33.148728 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.148704    4969 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-credential-socket\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:33.148728 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.148724    4969 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istio-data\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:33.148859 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.148734    4969 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istio-podinfo\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:33.148859 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.148743    4969 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istio-envoy\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:33.148859 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.148751    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4bvz\" (UniqueName: \"kubernetes.io/projected/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-kube-api-access-l4bvz\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:33.148859 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:33.148759    4969 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec-istio-token\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:24:34.036984 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:34.036951    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"
Nov 04 02:24:34.256423 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:34.256391    4969 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"]
Nov 04 02:24:34.261042 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:24:34.261011    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:24:34.305226 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:34.305168    4969 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-9adedd-openshift-default-bb77bbbc9-j7fjz"]
Nov 04 02:24:36.202439 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:36.202406    4969 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec" path="/var/lib/kubelet/pods/c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec/volumes"
Nov 04 02:24:42.346382 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:42.346333    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-7cf899df5-cdbnv_31d608ea-2e0c-4bfc-b80a-370eb6a7ad01/manager/0.log"
Nov 04 02:24:42.691027 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:42.690929    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-jp7lh_32f36ce2-d4d6-4b82-9ffe-ff5796462611/manager/0.log"
Nov 04 02:24:43.765308 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:43.765232    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-f759c744c-cr7gd_1bedd7c8-c5a9-419d-90bc-de1428f21eb3/manager/1.log"
Nov 04 02:24:44.074119 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:44.074012    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-52dc4_46cceba3-d979-4401-b4f3-decd198016e8/server/0.log"
Nov 04 02:24:45.725776 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:45.725733    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-f759c744c-cr7gd_1bedd7c8-c5a9-419d-90bc-de1428f21eb3/manager/0.log"
Nov 04 02:24:46.414261 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:46.414222    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-55445df7f8-pvcfj_e2e18c33-58c3-48a3-8c25-bd1adbcbfa59/maas-api/0.log"
Nov 04 02:24:52.215823 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:52.215790    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-58bcb58bd7-lpkwf_9627a478-94ef-41ca-b965-3982e66f502f/authorino/0.log"
Nov 04 02:24:52.900938 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:52.900910    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-dzk6c_de6afddf-ef49-434e-a869-4ff600c01ee5/manager/0.log"
Nov 04 02:24:53.803609 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:53.803561    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-sc7zj_595c2698-cf72-4c37-8096-bf3dac4c20b7/manager/0.log"
Nov 04 02:24:56.097934 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:56.097904    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56b54d794b-mf6t9_f71e7139-be33-407f-ad04-ecd9ee83ea48/kube-auth-proxy/0.log"
Nov 04 02:24:56.591206 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:56.591134    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6cc458d887-8bglg_6d4bc677-6239-47b2-92c1-1924e9cd1b54/istio-proxy/1.log"
Nov 04 02:24:57.655700 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:57.655670    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-7vwj2_20254f43-5d9a-4d1a-9b14-6fe6580dd397/payload-pre-processing/0.log"
Nov 04 02:24:57.903344 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:57.903315    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-pl7h2_90527deb-17f0-45d8-846a-99add8a3a440/payload-processing/0.log"
Nov 04 02:24:58.148458 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:58.148416    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6595f8bf85-dmj26_32699b90-bd7d-43f5-bdcc-53de9da6f244/router/0.log"
Nov 04 02:24:58.863586 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:58.863555    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6cc458d887-8bglg_6d4bc677-6239-47b2-92c1-1924e9cd1b54/istio-proxy/0.log"
Nov 04 02:24:59.792493 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:24:59.792464    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-p6m4m_0efb04e0-fa93-4504-9a40-c48f5451d9cd/main/0.log"
Nov 04 02:25:00.394933 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:00.394897    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:25:00.643792 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:00.643736    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm_759da135-51be-4cc8-bd82-a32afb1f8456/main/0.log"
Nov 04 02:25:01.188825 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:01.188793    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq_8c591d3b-6aba-4492-8409-54a8c36d1b85/main/0.log"
Nov 04 02:25:02.777618 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:02.777589    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-58bcb58bd7-lpkwf_9627a478-94ef-41ca-b965-3982e66f502f/authorino/0.log"
Nov 04 02:25:04.260046 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:25:04.260010    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:25:13.187715 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:13.187680    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-7cf899df5-cdbnv_31d608ea-2e0c-4bfc-b80a-370eb6a7ad01/manager/0.log"
Nov 04 02:25:13.542328 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:13.542300    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-jp7lh_32f36ce2-d4d6-4b82-9ffe-ff5796462611/manager/0.log"
Nov 04 02:25:14.471428 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:14.471397    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-f759c744c-cr7gd_1bedd7c8-c5a9-419d-90bc-de1428f21eb3/manager/1.log"
Nov 04 02:25:14.764910 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:14.764890    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-52dc4_46cceba3-d979-4401-b4f3-decd198016e8/server/0.log"
Nov 04 02:25:16.137345 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:16.137314    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-55445df7f8-pvcfj_e2e18c33-58c3-48a3-8c25-bd1adbcbfa59/maas-api/0.log"
Nov 04 02:25:19.268108 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:19.268058    4969 generic.go:356] "Generic (PLEG): container finished" podID="1a49f2af-8cb1-42fe-a537-4723b2437491" containerID="ad5ac6daa13a6c0cdb9f103fe3af955ca4531298ca0334ba3f5e8f0b239442d2" exitCode=0
Nov 04 02:25:19.268507 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:19.268128    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-p42b2-dzdbc" event={"ID":"1a49f2af-8cb1-42fe-a537-4723b2437491","Type":"ContainerDied","Data":"ad5ac6daa13a6c0cdb9f103fe3af955ca4531298ca0334ba3f5e8f0b239442d2"}
Nov 04 02:25:20.411848 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:20.411822    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-p42b2-dzdbc"
Nov 04 02:25:20.456760 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:20.456735    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a49f2af-8cb1-42fe-a537-4723b2437491-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a49f2af-8cb1-42fe-a537-4723b2437491-service-ca-bundle\") pod \"1a49f2af-8cb1-42fe-a537-4723b2437491\" (UID: \"1a49f2af-8cb1-42fe-a537-4723b2437491\") "
Nov 04 02:25:20.456876 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:20.456767    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a49f2af-8cb1-42fe-a537-4723b2437491-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a49f2af-8cb1-42fe-a537-4723b2437491-trusted-ca-bundle\") pod \"1a49f2af-8cb1-42fe-a537-4723b2437491\" (UID: \"1a49f2af-8cb1-42fe-a537-4723b2437491\") "
Nov 04 02:25:20.456876 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:20.456787    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a49f2af-8cb1-42fe-a537-4723b2437491-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/1a49f2af-8cb1-42fe-a537-4723b2437491-archives-path\") pod \"1a49f2af-8cb1-42fe-a537-4723b2437491\" (UID: \"1a49f2af-8cb1-42fe-a537-4723b2437491\") "
Nov 04 02:25:20.456876 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:20.456818    4969 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a49f2af-8cb1-42fe-a537-4723b2437491-kube-api-access-rc9k5\" (UniqueName: \"kubernetes.io/projected/1a49f2af-8cb1-42fe-a537-4723b2437491-kube-api-access-rc9k5\") pod \"1a49f2af-8cb1-42fe-a537-4723b2437491\" (UID: \"1a49f2af-8cb1-42fe-a537-4723b2437491\") "
Nov 04 02:25:20.457175 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:20.457151    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a49f2af-8cb1-42fe-a537-4723b2437491-service-ca-bundle" pod "1a49f2af-8cb1-42fe-a537-4723b2437491" (UID: "1a49f2af-8cb1-42fe-a537-4723b2437491"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:25:20.457175 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:20.457160    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a49f2af-8cb1-42fe-a537-4723b2437491-archives-path" pod "1a49f2af-8cb1-42fe-a537-4723b2437491" (UID: "1a49f2af-8cb1-42fe-a537-4723b2437491"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 02:25:20.457256 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:20.457183    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a49f2af-8cb1-42fe-a537-4723b2437491-trusted-ca-bundle" pod "1a49f2af-8cb1-42fe-a537-4723b2437491" (UID: "1a49f2af-8cb1-42fe-a537-4723b2437491"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 02:25:20.457293 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:20.457261    4969 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a49f2af-8cb1-42fe-a537-4723b2437491-service-ca-bundle\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:25:20.457293 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:20.457273    4969 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a49f2af-8cb1-42fe-a537-4723b2437491-trusted-ca-bundle\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:25:20.457293 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:20.457283    4969 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/1a49f2af-8cb1-42fe-a537-4723b2437491-archives-path\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:25:20.458797 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:20.458775    4969 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a49f2af-8cb1-42fe-a537-4723b2437491-kube-api-access-rc9k5" pod "1a49f2af-8cb1-42fe-a537-4723b2437491" (UID: "1a49f2af-8cb1-42fe-a537-4723b2437491"). InnerVolumeSpecName "kube-api-access-rc9k5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 02:25:20.557772 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:20.557713    4969 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rc9k5\" (UniqueName: \"kubernetes.io/projected/1a49f2af-8cb1-42fe-a537-4723b2437491-kube-api-access-rc9k5\") on node \"ip-10-0-136-63.us-west-1.compute.internal\" DevicePath \"\""
Nov 04 02:25:21.253224 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:21.253192    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-58bcb58bd7-lpkwf_9627a478-94ef-41ca-b965-3982e66f502f/authorino/0.log"
Nov 04 02:25:21.279042 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:21.279012    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-p42b2-dzdbc" event={"ID":"1a49f2af-8cb1-42fe-a537-4723b2437491","Type":"ContainerDied","Data":"0d27a6a2dda771d1702594a778c97b87cdc10ad73ce2986dd985f021b602a624"}
Nov 04 02:25:21.279042 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:21.279040    4969 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-p42b2-dzdbc"
Nov 04 02:25:21.279252 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:21.279054    4969 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d27a6a2dda771d1702594a778c97b87cdc10ad73ce2986dd985f021b602a624"
Nov 04 02:25:21.980908 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:21.980879    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-dzk6c_de6afddf-ef49-434e-a869-4ff600c01ee5/manager/0.log"
Nov 04 02:25:22.267892 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:22.267866    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:25:22.444728 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:22.444701    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:25:22.502042 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:22.502016    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-sc7zj_595c2698-cf72-4c37-8096-bf3dac4c20b7/manager/0.log"
Nov 04 02:25:24.533507 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:24.533476    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56b54d794b-mf6t9_f71e7139-be33-407f-ad04-ecd9ee83ea48/kube-auth-proxy/0.log"
Nov 04 02:25:25.040794 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:25.040764    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6cc458d887-8bglg_6d4bc677-6239-47b2-92c1-1924e9cd1b54/istio-proxy/1.log"
Nov 04 02:25:26.122754 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:26.122719    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-7vwj2_20254f43-5d9a-4d1a-9b14-6fe6580dd397/payload-pre-processing/0.log"
Nov 04 02:25:26.375349 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:26.375273    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-pl7h2_90527deb-17f0-45d8-846a-99add8a3a440/payload-processing/0.log"
Nov 04 02:25:26.630892 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:26.630817    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6595f8bf85-dmj26_32699b90-bd7d-43f5-bdcc-53de9da6f244/router/0.log"
Nov 04 02:25:27.542551 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:27.542521    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-p6m4m_0efb04e0-fa93-4504-9a40-c48f5451d9cd/main/0.log"
Nov 04 02:25:28.351827 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:28.351800    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8dhslm_759da135-51be-4cc8-bd82-a32afb1f8456/main/0.log"
Nov 04 02:25:28.852433 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:28.852405    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-zrltq_8c591d3b-6aba-4492-8409-54a8c36d1b85/main/0.log"
Nov 04 02:25:34.259731 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:25:34.259694    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:25:39.250394 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:39.250342    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:25:40.491873 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:40.491840    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:25:56.617392 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:56.617353    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-f759c744c-cr7gd_1bedd7c8-c5a9-419d-90bc-de1428f21eb3/manager/1.log"
Nov 04 02:25:57.148109 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:57.148071    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-f759c744c-cr7gd_1bedd7c8-c5a9-419d-90bc-de1428f21eb3/manager/1.log"
Nov 04 02:25:57.911022 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:57.910990    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-pl7h2_90527deb-17f0-45d8-846a-99add8a3a440/payload-processing/0.log"
Nov 04 02:25:58.142402 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:25:58.142375    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-pl7h2_90527deb-17f0-45d8-846a-99add8a3a440/payload-processing/0.log"
Nov 04 02:26:00.755666 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:00.755624    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:26:04.259908 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:26:04.259870    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:26:04.351869 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:04.351837    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jcqmp_16f00163-6ec3-4c10-81e4-c8a501e15e82/console-operator/2.log"
Nov 04 02:26:04.354837 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:04.354818    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jcqmp_16f00163-6ec3-4c10-81e4-c8a501e15e82/console-operator/2.log"
Nov 04 02:26:32.120979 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:32.120946    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-f759c744c-cr7gd_1bedd7c8-c5a9-419d-90bc-de1428f21eb3/manager/1.log"
Nov 04 02:26:32.630513 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:32.630481    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-f759c744c-cr7gd_1bedd7c8-c5a9-419d-90bc-de1428f21eb3/manager/1.log"
Nov 04 02:26:33.448847 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:33.448811    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-pl7h2_90527deb-17f0-45d8-846a-99add8a3a440/payload-processing/0.log"
Nov 04 02:26:33.661597 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:33.661567    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-pl7h2_90527deb-17f0-45d8-846a-99add8a3a440/payload-processing/0.log"
Nov 04 02:26:34.020361 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.020323    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pnj78/must-gather-75r72"]
Nov 04 02:26:34.020828 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.020809    4969 cpu_manager.go:414] "containerMap: removing container" podUID="ea0b94ed-20c6-4b3a-abd6-e39992324ec6" containerName="maas-api"
Nov 04 02:26:34.020828 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.020828    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ea0b94ed-20c6-4b3a-abd6-e39992324ec6" containerName="maas-api"
Nov 04 02:26:34.020963 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.020852    4969 cpu_manager.go:414] "containerMap: removing container" podUID="1a49f2af-8cb1-42fe-a537-4723b2437491" containerName="insights-gathering"
Nov 04 02:26:34.020963 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.020857    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a49f2af-8cb1-42fe-a537-4723b2437491" containerName="insights-gathering"
Nov 04 02:26:34.020963 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.020873    4969 cpu_manager.go:414] "containerMap: removing container" podUID="c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec" containerName="istio-proxy"
Nov 04 02:26:34.020963 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.020879    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec" containerName="istio-proxy"
Nov 04 02:26:34.020963 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.020894    4969 cpu_manager.go:414] "containerMap: removing container" podUID="4e50fee5-a7b8-436b-8685-ff90f478d369" containerName="istio-proxy"
Nov 04 02:26:34.020963 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.020898    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4e50fee5-a7b8-436b-8685-ff90f478d369" containerName="istio-proxy"
Nov 04 02:26:34.020963 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.020904    4969 cpu_manager.go:414] "containerMap: removing container" podUID="0ba3e718-6552-472a-8002-259a7ffc4b89" containerName="istio-proxy"
Nov 04 02:26:34.020963 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.020909    4969 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0ba3e718-6552-472a-8002-259a7ffc4b89" containerName="istio-proxy"
Nov 04 02:26:34.021230 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.020971    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea0b94ed-20c6-4b3a-abd6-e39992324ec6" containerName="maas-api"
Nov 04 02:26:34.021230 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.020983    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a49f2af-8cb1-42fe-a537-4723b2437491" containerName="insights-gathering"
Nov 04 02:26:34.021230 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.020990    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="c9f1b6e2-7dd2-41ac-8c57-f35f0c2a4eec" containerName="istio-proxy"
Nov 04 02:26:34.021230 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.020997    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="4e50fee5-a7b8-436b-8685-ff90f478d369" containerName="istio-proxy"
Nov 04 02:26:34.021230 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.021004    4969 memory_manager.go:365] "RemoveStaleState removing state" podUID="0ba3e718-6552-472a-8002-259a7ffc4b89" containerName="istio-proxy"
Nov 04 02:26:34.024290 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.024271    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pnj78/must-gather-75r72"
Nov 04 02:26:34.052972 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.052950    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:26:34.054229 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.054209    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6f026230-5dfb-429d-9121-89712ae8ebae-must-gather-output\") pod \"must-gather-75r72\" (UID: \"6f026230-5dfb-429d-9121-89712ae8ebae\") " pod="openshift-must-gather-pnj78/must-gather-75r72"
Nov 04 02:26:34.054292 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.054237    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tr9v9\" (UniqueName: \"kubernetes.io/projected/6f026230-5dfb-429d-9121-89712ae8ebae-kube-api-access-tr9v9\") pod \"must-gather-75r72\" (UID: \"6f026230-5dfb-429d-9121-89712ae8ebae\") " pod="openshift-must-gather-pnj78/must-gather-75r72"
Nov 04 02:26:34.128409 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.128384    4969 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pnj78\"/\"default-dockercfg-k64b5\""
Nov 04 02:26:34.128409 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.128408    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pnj78\"/\"openshift-service-ca.crt\""
Nov 04 02:26:34.128606 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.128441    4969 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pnj78\"/\"kube-root-ca.crt\""
Nov 04 02:26:34.130634 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.130610    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pnj78/must-gather-75r72"]
Nov 04 02:26:34.155110 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.155067    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6f026230-5dfb-429d-9121-89712ae8ebae-must-gather-output\") pod \"must-gather-75r72\" (UID: \"6f026230-5dfb-429d-9121-89712ae8ebae\") " pod="openshift-must-gather-pnj78/must-gather-75r72"
Nov 04 02:26:34.155216 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.155116    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr9v9\" (UniqueName: \"kubernetes.io/projected/6f026230-5dfb-429d-9121-89712ae8ebae-kube-api-access-tr9v9\") pod \"must-gather-75r72\" (UID: \"6f026230-5dfb-429d-9121-89712ae8ebae\") " pod="openshift-must-gather-pnj78/must-gather-75r72"
Nov 04 02:26:34.155426 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.155407    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6f026230-5dfb-429d-9121-89712ae8ebae-must-gather-output\") pod \"must-gather-75r72\" (UID: \"6f026230-5dfb-429d-9121-89712ae8ebae\") " pod="openshift-must-gather-pnj78/must-gather-75r72"
Nov 04 02:26:34.215544 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.215525    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tr9v9\" (UniqueName: \"kubernetes.io/projected/6f026230-5dfb-429d-9121-89712ae8ebae-kube-api-access-tr9v9\") pod \"must-gather-75r72\" (UID: \"6f026230-5dfb-429d-9121-89712ae8ebae\") " pod="openshift-must-gather-pnj78/must-gather-75r72"
Nov 04 02:26:34.231293 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.231250    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:26:34.259749 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:26:34.259723    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:26:34.338388 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.338329    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pnj78/must-gather-75r72"
Nov 04 02:26:34.711109 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:34.711063    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pnj78/must-gather-75r72"]
Nov 04 02:26:34.719073 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:26:34.719049    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f026230_5dfb_429d_9121_89712ae8ebae.slice/crio-1270de6620998afc27f7fe021db24ce09e23d07cbb6eed68de4d44f59f66e320 WatchSource:0}: Error finding container 1270de6620998afc27f7fe021db24ce09e23d07cbb6eed68de4d44f59f66e320: Status 404 returned error can't find the container with id 1270de6620998afc27f7fe021db24ce09e23d07cbb6eed68de4d44f59f66e320
Nov 04 02:26:35.659843 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:35.659806    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pnj78/must-gather-75r72" event={"ID":"6f026230-5dfb-429d-9121-89712ae8ebae","Type":"ContainerStarted","Data":"1270de6620998afc27f7fe021db24ce09e23d07cbb6eed68de4d44f59f66e320"}
Nov 04 02:26:36.666793 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:36.666753    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pnj78/must-gather-75r72" event={"ID":"6f026230-5dfb-429d-9121-89712ae8ebae","Type":"ContainerStarted","Data":"b93831d511916281921c7b38270dc62acdda3fc1834107312f282283c0002c88"}
Nov 04 02:26:36.667205 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:36.666806    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pnj78/must-gather-75r72" event={"ID":"6f026230-5dfb-429d-9121-89712ae8ebae","Type":"ContainerStarted","Data":"51b755d5bfa48914a332dc1b6f40816fa409d0ca812a68f76eb2e347a107b771"}
Nov 04 02:26:36.783191 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:36.783137    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-pnj78/must-gather-75r72" podStartSLOduration=2.47865286 podStartE2EDuration="3.783122882s" totalImagesPullingTime="1.304470022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:26:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:26:36.781029967 +0000 UTC m=+1232.707252600" watchObservedRunningTime="2026-11-04 02:26:36.783122882 +0000 UTC m=+1232.709345515"
Nov 04 02:26:38.488988 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:38.488948    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:26:38.672024 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:38.671991    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:26:43.028008 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:43.027983    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-b4wjr_e164905d-5505-4d20-af2e-b8acd370bf18/global-pull-secret-syncer/0.log"
Nov 04 02:26:44.869115 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:44.865847    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-2mklg_abdfaf94-b721-4ed4-8671-416e694df5bb/connection-checker/0.log"
Nov 04 02:26:45.296114 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:45.295892    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-d27bw_1597f8a8-5b13-4076-9d95-c494cf3ad5dc/connection-checker/0.log"
Nov 04 02:26:45.729164 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:45.726501    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-pn5gm_b2156825-c0d8-48a4-a90f-2340e98a13e3/connection-checker/0.log"
Nov 04 02:26:46.218385 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:46.218345    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log"
Nov 04 02:26:48.047032 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:48.046999    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-63.us-west-1.compute.internal_5dd9721b3a00168bde79d6cad3060599/haproxy/0.log"
Nov 04 02:26:51.863582 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:51.863543    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-58bcb58bd7-lpkwf_9627a478-94ef-41ca-b965-3982e66f502f/authorino/0.log"
Nov 04 02:26:52.609592 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:52.609561    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-dzk6c_de6afddf-ef49-434e-a869-4ff600c01ee5/manager/0.log"
Nov 04 02:26:52.804161 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:52.804123    4969 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pnj78/perf-node-gather-daemonset-mtpqf"]
Nov 04 02:26:52.808628 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:52.808599    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-mtpqf"
Nov 04 02:26:52.900209 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:52.900118    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pnj78/perf-node-gather-daemonset-mtpqf"]
Nov 04 02:26:52.927688 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:52.927661    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0980b511-8830-41c0-84d3-4dd07672ad4b-lib-modules\") pod \"perf-node-gather-daemonset-mtpqf\" (UID: \"0980b511-8830-41c0-84d3-4dd07672ad4b\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-mtpqf"
Nov 04 02:26:52.927861 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:52.927705    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjvvk\" (UniqueName: \"kubernetes.io/projected/0980b511-8830-41c0-84d3-4dd07672ad4b-kube-api-access-zjvvk\") pod \"perf-node-gather-daemonset-mtpqf\" (UID: \"0980b511-8830-41c0-84d3-4dd07672ad4b\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-mtpqf"
Nov 04 02:26:52.927861 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:52.927731    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0980b511-8830-41c0-84d3-4dd07672ad4b-proc\") pod \"perf-node-gather-daemonset-mtpqf\" (UID: \"0980b511-8830-41c0-84d3-4dd07672ad4b\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-mtpqf"
Nov 04 02:26:52.927861 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:52.927812    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0980b511-8830-41c0-84d3-4dd07672ad4b-podres\") pod \"perf-node-gather-daemonset-mtpqf\" (UID: \"0980b511-8830-41c0-84d3-4dd07672ad4b\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-mtpqf"
Nov 04 02:26:52.927861 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:52.927850    4969 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0980b511-8830-41c0-84d3-4dd07672ad4b-sys\") pod \"perf-node-gather-daemonset-mtpqf\" (UID: \"0980b511-8830-41c0-84d3-4dd07672ad4b\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-mtpqf"
Nov 04 02:26:53.029208 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:53.029177    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0980b511-8830-41c0-84d3-4dd07672ad4b-lib-modules\") pod \"perf-node-gather-daemonset-mtpqf\" (UID: \"0980b511-8830-41c0-84d3-4dd07672ad4b\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-mtpqf"
Nov 04 02:26:53.029382 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:53.029219    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjvvk\" (UniqueName: \"kubernetes.io/projected/0980b511-8830-41c0-84d3-4dd07672ad4b-kube-api-access-zjvvk\") pod \"perf-node-gather-daemonset-mtpqf\" (UID: \"0980b511-8830-41c0-84d3-4dd07672ad4b\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-mtpqf"
Nov 04 02:26:53.029382 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:53.029246    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0980b511-8830-41c0-84d3-4dd07672ad4b-proc\") pod \"perf-node-gather-daemonset-mtpqf\" (UID: \"0980b511-8830-41c0-84d3-4dd07672ad4b\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-mtpqf"
Nov 04 02:26:53.029382 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:53.029278    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0980b511-8830-41c0-84d3-4dd07672ad4b-podres\") pod \"perf-node-gather-daemonset-mtpqf\" (UID: \"0980b511-8830-41c0-84d3-4dd07672ad4b\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-mtpqf"
Nov 04 02:26:53.029382 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:53.029314    4969 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0980b511-8830-41c0-84d3-4dd07672ad4b-sys\") pod \"perf-node-gather-daemonset-mtpqf\" (UID: \"0980b511-8830-41c0-84d3-4dd07672ad4b\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-mtpqf"
Nov 04 02:26:53.029382 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:53.029332    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0980b511-8830-41c0-84d3-4dd07672ad4b-lib-modules\") pod \"perf-node-gather-daemonset-mtpqf\" (UID: \"0980b511-8830-41c0-84d3-4dd07672ad4b\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-mtpqf"
Nov 04 02:26:53.029382 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:53.029378    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0980b511-8830-41c0-84d3-4dd07672ad4b-proc\") pod \"perf-node-gather-daemonset-mtpqf\" (UID: \"0980b511-8830-41c0-84d3-4dd07672ad4b\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-mtpqf"
Nov 04 02:26:53.029612 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:53.029401    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0980b511-8830-41c0-84d3-4dd07672ad4b-podres\") pod \"perf-node-gather-daemonset-mtpqf\" (UID: \"0980b511-8830-41c0-84d3-4dd07672ad4b\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-mtpqf"
Nov 04 02:26:53.029612 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:53.029420    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0980b511-8830-41c0-84d3-4dd07672ad4b-sys\") pod \"perf-node-gather-daemonset-mtpqf\" (UID: \"0980b511-8830-41c0-84d3-4dd07672ad4b\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-mtpqf"
Nov 04 02:26:53.089842 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:53.089811    4969 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjvvk\" (UniqueName: \"kubernetes.io/projected/0980b511-8830-41c0-84d3-4dd07672ad4b-kube-api-access-zjvvk\") pod \"perf-node-gather-daemonset-mtpqf\" (UID: \"0980b511-8830-41c0-84d3-4dd07672ad4b\") " pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-mtpqf"
Nov 04 02:26:53.126850 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:53.126823    4969 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-mtpqf"
Nov 04 02:26:53.504009 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:53.503976    4969 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pnj78/perf-node-gather-daemonset-mtpqf"]
Nov 04 02:26:53.513841 ip-10-0-136-63 kubenswrapper[4969]: W1104 02:26:53.513807    4969 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0980b511_8830_41c0_84d3_4dd07672ad4b.slice/crio-cdf9cff4826a31c345d1e83b76f4f98454e226fc70f2ee454747df3197c73018 WatchSource:0}: Error finding container cdf9cff4826a31c345d1e83b76f4f98454e226fc70f2ee454747df3197c73018: Status 404 returned error can't find the container with id cdf9cff4826a31c345d1e83b76f4f98454e226fc70f2ee454747df3197c73018
Nov 04 02:26:53.708699 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:53.708666    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-sc7zj_595c2698-cf72-4c37-8096-bf3dac4c20b7/manager/0.log"
Nov 04 02:26:53.816108 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:53.816060    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-mtpqf" event={"ID":"0980b511-8830-41c0-84d3-4dd07672ad4b","Type":"ContainerStarted","Data":"7f3f2db307e9aeddb0031557f40aa8d1fb1a993311f96336aee7c6606a055903"}
Nov 04 02:26:53.816305 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:53.816145    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-mtpqf"
Nov 04 02:26:53.816305 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:53.816165    4969 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-mtpqf" event={"ID":"0980b511-8830-41c0-84d3-4dd07672ad4b","Type":"ContainerStarted","Data":"cdf9cff4826a31c345d1e83b76f4f98454e226fc70f2ee454747df3197c73018"}
Nov 04 02:26:53.933061 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:53.933004    4969 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-mtpqf" podStartSLOduration=1.932988454 podStartE2EDuration="1.932988454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:26:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:26:53.931640791 +0000 UTC m=+1249.857863426" watchObservedRunningTime="2026-11-04 02:26:53.932988454 +0000 UTC m=+1249.859211087"
Nov 04 02:26:59.832740 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:26:59.832708    4969 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pnj78/perf-node-gather-daemonset-mtpqf"
Nov 04 02:27:00.123741 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:27:00.123660    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-btjpg_63f7e9d4-d5e9-4115-9aa8-fc8a6aae5faa/cluster-monitoring-operator/0.log"
Nov 04 02:27:01.108101 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:27:01.108056    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024
Nov 04 02:27:04.261500 ip-10-0-136-63 kubenswrapper[4969]: E1104 02:27:04.261461    4969 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-hh6m5" containerName="extractor"
Nov 04 02:27:05.320712 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:27:05.320660    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tdjrt_067adedf-2b86-48dd-9005-47e0178b9824/node-exporter/0.log"
Nov 04 02:27:05.738381 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:27:05.738308    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tdjrt_067adedf-2b86-48dd-9005-47e0178b9824/kube-rbac-proxy/0.log"
Nov 04 02:27:06.158333 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:27:06.158302    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tdjrt_067adedf-2b86-48dd-9005-47e0178b9824/init-textfile/0.log"
Nov 04 02:27:10.910678 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:27:10.910646    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-kjgvd_1944cee5-2a0d-4978-badf-0471ae90f700/prometheus-operator/0.log"
Nov 04 02:27:11.332990 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:27:11.332925    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-kjgvd_1944cee5-2a0d-4978-badf-0471ae90f700/kube-rbac-proxy/0.log"
Nov 04 02:27:11.752522 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:27:11.752457    4969 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-xw9tf_48814638-7f4b-4276-9d0d-97b0d80f46cd/prometheus-operator-admission-webhook/0.log"
Nov 04 02:27:12.445386 ip-10-0-136-63 kubenswrapper[4969]: I1104 02:27:12.445355    4969 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-928nm_39235bb9-36b4-43e4-9161-043a3fb6a261/konnectivity-agent/0.log" limit=1024