Jul 10 22:55:37.550754 ip-10-0-141-239 systemd[1]: Starting Kubernetes Kubelet...
Jul 10 22:55:37.869025 ip-10-0-141-239 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 22:55:37.869025 ip-10-0-141-239 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 10 22:55:37.869025 ip-10-0-141-239 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 22:55:37.869025 ip-10-0-141-239 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 10 22:55:37.869025 ip-10-0-141-239 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 22:55:37.869787 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.869658    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 10 22:55:37.874732 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874706    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 22:55:37.874732 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874729    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 22:55:37.874732 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874733    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 22:55:37.874732 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874736    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 22:55:37.874861 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874739    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 22:55:37.874861 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874742    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 22:55:37.874861 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874745    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 22:55:37.874861 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874747    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 22:55:37.874861 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874750    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 22:55:37.874861 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874753    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 22:55:37.874861 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874755    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 22:55:37.874861 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874759    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 22:55:37.874861 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874762    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 22:55:37.874861 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874764    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 22:55:37.874861 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874767    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 22:55:37.874861 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874770    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 22:55:37.874861 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874773    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 22:55:37.874861 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874776    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 22:55:37.874861 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874778    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 22:55:37.874861 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874789    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 22:55:37.874861 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874792    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 22:55:37.874861 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874794    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 22:55:37.874861 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874797    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 22:55:37.874861 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874800    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 22:55:37.874861 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874803    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 22:55:37.875373 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874805    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 22:55:37.875373 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874808    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 22:55:37.875373 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874811    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 22:55:37.875373 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874814    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 22:55:37.875373 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874816    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 22:55:37.875373 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874819    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 22:55:37.875373 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874821    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 22:55:37.875373 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874824    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 22:55:37.875373 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874827    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 22:55:37.875373 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874830    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 22:55:37.875373 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874832    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 22:55:37.875373 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874835    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 22:55:37.875373 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874839    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 22:55:37.875373 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874844    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 22:55:37.875373 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874847    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 22:55:37.875373 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874850    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 22:55:37.875373 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874853    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 22:55:37.875373 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874856    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 22:55:37.875373 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874859    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 22:55:37.875880 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874861    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 22:55:37.875880 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874864    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 22:55:37.875880 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874867    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 22:55:37.875880 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874870    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 22:55:37.875880 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874873    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 22:55:37.875880 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874876    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 22:55:37.875880 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874878    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 22:55:37.875880 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874881    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 22:55:37.875880 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874884    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 22:55:37.875880 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874886    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 22:55:37.875880 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874889    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 22:55:37.875880 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874891    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 22:55:37.875880 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874893    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 22:55:37.875880 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874896    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 22:55:37.875880 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874899    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 22:55:37.875880 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874902    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 22:55:37.875880 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874904    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 22:55:37.875880 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874907    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 22:55:37.875880 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874909    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 22:55:37.876345 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874912    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 22:55:37.876345 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874915    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 22:55:37.876345 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874919    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 22:55:37.876345 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874922    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 22:55:37.876345 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874925    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 22:55:37.876345 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874927    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 22:55:37.876345 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874929    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 22:55:37.876345 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874932    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 22:55:37.876345 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874938    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 22:55:37.876345 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874941    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 22:55:37.876345 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874943    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 22:55:37.876345 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874946    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 22:55:37.876345 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874948    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 22:55:37.876345 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874951    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 22:55:37.876345 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874953    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 22:55:37.876345 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874956    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 22:55:37.876345 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874960    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 22:55:37.876345 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874962    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 22:55:37.876345 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874964    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 22:55:37.876345 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874967    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 22:55:37.876852 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874969    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 22:55:37.876852 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874971    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 22:55:37.876852 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874974    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 22:55:37.876852 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874976    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 22:55:37.876852 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874978    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 22:55:37.876852 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874981    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 22:55:37.876852 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874983    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 22:55:37.876852 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874987    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 22:55:37.876852 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874991    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 22:55:37.876852 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874993    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 22:55:37.876852 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874996    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 22:55:37.876852 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.874998    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 22:55:37.876852 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875000    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 22:55:37.876852 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875003    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 22:55:37.876852 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875006    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 22:55:37.876852 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875008    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 22:55:37.876852 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875781    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 22:55:37.876852 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875789    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 22:55:37.876852 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875792    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 22:55:37.877337 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875795    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 22:55:37.877337 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875797    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 22:55:37.877337 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875801    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 22:55:37.877337 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875804    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 22:55:37.877337 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875806    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 22:55:37.877337 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875809    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 22:55:37.877337 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875811    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 22:55:37.877337 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875814    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 22:55:37.877337 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875816    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 22:55:37.877337 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875819    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 22:55:37.877337 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875821    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 22:55:37.877337 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875823    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 22:55:37.877337 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875826    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 22:55:37.877337 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875828    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 22:55:37.877337 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875831    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 22:55:37.877337 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875834    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 22:55:37.877337 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875836    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 22:55:37.877337 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875839    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 22:55:37.877337 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875841    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 22:55:37.877337 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875844    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 22:55:37.877850 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875847    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 22:55:37.877850 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875850    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 22:55:37.877850 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875854    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 22:55:37.877850 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875858    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 22:55:37.877850 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875861    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 22:55:37.877850 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875863    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 22:55:37.877850 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875877    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 22:55:37.877850 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875882    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 22:55:37.877850 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875885    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 22:55:37.877850 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875888    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 22:55:37.877850 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875890    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 22:55:37.877850 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875893    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 22:55:37.877850 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875897    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 22:55:37.877850 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875899    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 22:55:37.877850 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875902    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 22:55:37.877850 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875905    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 22:55:37.877850 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875908    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 22:55:37.877850 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875910    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 22:55:37.877850 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875914    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 22:55:37.878310 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875917    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 22:55:37.878310 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875920    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 22:55:37.878310 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875923    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 22:55:37.878310 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875926    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 22:55:37.878310 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875928    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 22:55:37.878310 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875931    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 22:55:37.878310 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875934    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 22:55:37.878310 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875937    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 22:55:37.878310 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875940    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 22:55:37.878310 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875942    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 22:55:37.878310 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875945    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 22:55:37.878310 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875947    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 22:55:37.878310 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875950    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 22:55:37.878310 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875953    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 22:55:37.878310 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875956    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 22:55:37.878310 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875958    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 22:55:37.878310 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875961    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 22:55:37.878310 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875963    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 22:55:37.878310 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875965    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 22:55:37.878310 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875968    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 22:55:37.878820 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875971    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 22:55:37.878820 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875973    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 22:55:37.878820 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875977    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 22:55:37.878820 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875981    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 22:55:37.878820 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875984    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 22:55:37.878820 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875986    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 22:55:37.878820 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875989    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 22:55:37.878820 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875991    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 22:55:37.878820 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875994    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 22:55:37.878820 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875997    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 22:55:37.878820 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.875999    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 22:55:37.878820 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876001    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 22:55:37.878820 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876004    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 22:55:37.878820 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876006    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 22:55:37.878820 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876010    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 22:55:37.878820 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876012    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 22:55:37.878820 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876015    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 22:55:37.878820 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876017    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 22:55:37.878820 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876020    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 22:55:37.879286 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876022    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 22:55:37.879286 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876025    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 22:55:37.879286 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876027    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 22:55:37.879286 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876029    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 22:55:37.879286 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876032    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 22:55:37.879286 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876035    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 22:55:37.879286 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876037    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 22:55:37.879286 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876041    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 22:55:37.879286 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876043    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 22:55:37.879286 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876046    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 22:55:37.879286 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876048    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 22:55:37.879286 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876050    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 22:55:37.879286 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876053    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 22:55:37.879286 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876056    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 22:55:37.879286 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876058    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 22:55:37.879286 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876061    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 22:55:37.879286 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876064    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 22:55:37.879286 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876066    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 22:55:37.879286 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876549    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 22:55:37.879286 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876555    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 22:55:37.879913 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876558    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 22:55:37.879913 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876561    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 22:55:37.879913 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876569    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 22:55:37.879913 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876572    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 22:55:37.879913 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876575    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 22:55:37.879913 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876577    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 22:55:37.879913 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876580    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 22:55:37.879913 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876582    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 22:55:37.879913 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876584    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 22:55:37.879913 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876588    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 22:55:37.879913 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876591    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 22:55:37.879913 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876594    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 22:55:37.879913 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876596    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 22:55:37.879913 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876599    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 22:55:37.879913 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876601    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 22:55:37.879913 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876604    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 22:55:37.879913 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876606    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 22:55:37.879913 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876608    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 22:55:37.879913 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876611    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 22:55:37.879913 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876614    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 22:55:37.880408 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876616    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 22:55:37.880408 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876618    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 22:55:37.880408 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876621    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 22:55:37.880408 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876623    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 22:55:37.880408 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876626    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 22:55:37.880408 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876629    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 22:55:37.880408 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876631    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 22:55:37.880408 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876634    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 22:55:37.880408 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876636    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 22:55:37.880408 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876639    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 22:55:37.880408 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876641    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 22:55:37.880408 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876643    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 22:55:37.880408 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876646    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 22:55:37.880408 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876648    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 22:55:37.880408 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876651    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 22:55:37.880408 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876659    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 22:55:37.880408 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876662    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 22:55:37.880408 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876664    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 22:55:37.880408 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876666    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 22:55:37.880408 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876669    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 22:55:37.880914 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876672    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 22:55:37.880914 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876674    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 22:55:37.880914 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876677    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 22:55:37.880914 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876679    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 22:55:37.880914 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876682    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 22:55:37.880914 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876690    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 22:55:37.880914 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876693    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 22:55:37.880914 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876695    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 22:55:37.880914 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876698    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 22:55:37.880914 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876700    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 22:55:37.880914 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876703    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 22:55:37.880914 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876706    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 22:55:37.880914 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876708    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 22:55:37.880914 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876711    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 22:55:37.880914 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876715    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 22:55:37.880914 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876719    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 22:55:37.880914 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876724    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 22:55:37.880914 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876727    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 22:55:37.881366 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876730    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 22:55:37.881366 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876733    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 22:55:37.881366 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876736    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 22:55:37.881366 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876738    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 22:55:37.881366 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876741    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 22:55:37.881366 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876743    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 22:55:37.881366 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876745    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 22:55:37.881366 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876748    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 22:55:37.881366 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876750    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 22:55:37.881366 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876753    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 22:55:37.881366 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876762    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 22:55:37.881366 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876765    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 22:55:37.881366 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876768    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 22:55:37.881366 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876770    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 22:55:37.881366 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876773    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 22:55:37.881366 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876779    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 22:55:37.881366 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876781    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 22:55:37.881366 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876784    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 22:55:37.881366 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876786    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 22:55:37.881366 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876789    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 22:55:37.881878 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876792    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 22:55:37.881878 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876794    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 22:55:37.881878 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876796    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 22:55:37.881878 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876799    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 22:55:37.881878 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876801    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 22:55:37.881878 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876804    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 22:55:37.881878 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876806    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 22:55:37.881878 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876809    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 22:55:37.881878 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876811    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 22:55:37.881878 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876814    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 22:55:37.881878 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876817    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 22:55:37.881878 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876819    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 22:55:37.881878 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876822    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 22:55:37.881878 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876824    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 22:55:37.881878 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876827    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 22:55:37.881878 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876829    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 22:55:37.881878 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876831    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 22:55:37.881878 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876834    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 22:55:37.881878 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.876836    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 22:55:37.881878 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.876922    2563 flags.go:64] FLAG: --address="0.0.0.0"
Jul 10 22:55:37.881878 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.876930    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 10 22:55:37.882434 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.876943    2563 flags.go:64] FLAG: --anonymous-auth="true"
Jul 10 22:55:37.882434 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.876947    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 10 22:55:37.882434 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.876967    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 10 22:55:37.882434 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.876972    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 10 22:55:37.882434 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.876976    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 10 22:55:37.882434 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.876980    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 10 22:55:37.882434 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.876984    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 10 22:55:37.882434 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.876987    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 10 22:55:37.882434 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.876991    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 10 22:55:37.882434 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.876994    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 10 22:55:37.882434 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.876997    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 10 22:55:37.882434 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877000    2563 flags.go:64] FLAG: --cgroup-root=""
Jul 10 22:55:37.882434 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877003    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 10 22:55:37.882434 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877007    2563 flags.go:64] FLAG: --client-ca-file=""
Jul 10 22:55:37.882434 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877009    2563 flags.go:64] FLAG: --cloud-provider="external"
Jul 10 22:55:37.882434 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877012    2563 flags.go:64] FLAG: --cluster-dns="[]"
Jul 10 22:55:37.882434 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877022    2563 flags.go:64] FLAG: --cluster-domain=""
Jul 10 22:55:37.882434 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877025    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 10 22:55:37.882434 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877028    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 10 22:55:37.882434 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877031    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 10 22:55:37.882434 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877034    2563 flags.go:64] FLAG: --container-log-max-files="5"
Jul 10 22:55:37.882434 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877038    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 10 22:55:37.882434 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877041    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 10 22:55:37.882989 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877044    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 10 22:55:37.882989 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877047    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 10 22:55:37.882989 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877050    2563 flags.go:64] FLAG: --contention-profiling="false"
Jul 10 22:55:37.882989 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877053    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 10 22:55:37.882989 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877056    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 10 22:55:37.882989 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877059    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 10 22:55:37.882989 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877061    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 10 22:55:37.882989 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877065    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 10 22:55:37.882989 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877068    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 10 22:55:37.882989 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877071    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 10 22:55:37.882989 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877073    2563 flags.go:64] FLAG: --enable-load-reader="false"
Jul 10 22:55:37.882989 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877076    2563 flags.go:64] FLAG: --enable-server="true"
Jul 10 22:55:37.882989 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877085    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 10 22:55:37.882989 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877091    2563 flags.go:64] FLAG: --event-burst="100"
Jul 10 22:55:37.882989 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877095    2563 flags.go:64] FLAG: --event-qps="50"
Jul 10 22:55:37.882989 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877098    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 10 22:55:37.882989 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877101    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 10 22:55:37.882989 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877104    2563 flags.go:64] FLAG: --eviction-hard=""
Jul 10 22:55:37.882989 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877111    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 10 22:55:37.882989 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877114    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 10 22:55:37.882989 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877117    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 10 22:55:37.882989 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877120    2563 flags.go:64] FLAG: --eviction-soft=""
Jul 10 22:55:37.882989 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877122    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 10 22:55:37.882989 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877125    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 10 22:55:37.882989 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877128    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 10 22:55:37.883667 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877130    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 10 22:55:37.883667 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877134    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 10 22:55:37.883667 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877136    2563 flags.go:64] FLAG: --fail-swap-on="true"
Jul 10 22:55:37.883667 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877139    2563 flags.go:64] FLAG: --feature-gates=""
Jul 10 22:55:37.883667 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877143    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 10 22:55:37.883667 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877146    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 10 22:55:37.883667 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877149    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 10 22:55:37.883667 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877152    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 10 22:55:37.883667 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877155    2563 flags.go:64] FLAG: --healthz-port="10248"
Jul 10 22:55:37.883667 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877158    2563 flags.go:64] FLAG: --help="false"
Jul 10 22:55:37.883667 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877160    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-141-239.ec2.internal"
Jul 10 22:55:37.883667 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877163    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 10 22:55:37.883667 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877166    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 10 22:55:37.883667 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877169    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 10 22:55:37.883667 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877172    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 10 22:55:37.883667 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877178    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 10 22:55:37.883667 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877181    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 10 22:55:37.883667 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877183    2563 flags.go:64] FLAG: --image-service-endpoint=""
Jul 10 22:55:37.883667 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877186    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 10 22:55:37.883667 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877189    2563 flags.go:64] FLAG: --kube-api-burst="100"
Jul 10 22:55:37.883667 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877197    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 10 22:55:37.883667 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877201    2563 flags.go:64] FLAG: --kube-api-qps="50"
Jul 10 22:55:37.883667 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877203    2563 flags.go:64] FLAG: --kube-reserved=""
Jul 10 22:55:37.883667 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877206    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 10 22:55:37.884313 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877209    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 10 22:55:37.884313 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877212    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 10 22:55:37.884313 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877214    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 10 22:55:37.884313 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877217    2563 flags.go:64] FLAG: --lock-file=""
Jul 10 22:55:37.884313 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877220    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 10 22:55:37.884313 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877223    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 10 22:55:37.884313 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877226    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 10 22:55:37.884313 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877231    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 10 22:55:37.884313 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877233    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 10 22:55:37.884313 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877237    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 10 22:55:37.884313 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877240    2563 flags.go:64] FLAG: --logging-format="text"
Jul 10 22:55:37.884313 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877243    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 10 22:55:37.884313 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877246    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 10 22:55:37.884313 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877249    2563 flags.go:64] FLAG: --manifest-url=""
Jul 10 22:55:37.884313 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877251    2563 flags.go:64] FLAG: --manifest-url-header=""
Jul 10 22:55:37.884313 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877256    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 10 22:55:37.884313 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877259    2563 flags.go:64] FLAG: --max-open-files="1000000"
Jul 10 22:55:37.884313 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877263    2563 flags.go:64] FLAG: --max-pods="110"
Jul 10 22:55:37.884313 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877266    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 10 22:55:37.884313 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877269    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 10 22:55:37.884313 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877272    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 10 22:55:37.884313 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877275    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 10 22:55:37.884313 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877277    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 10 22:55:37.884313 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877280    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 10 22:55:37.884313 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877283    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 10 22:55:37.884949 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877290    2563 flags.go:64] FLAG: --node-status-max-images="50"
Jul 10 22:55:37.884949 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877293    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 10 22:55:37.884949 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877296    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 10 22:55:37.884949 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877299    2563 flags.go:64] FLAG: --pod-cidr=""
Jul 10 22:55:37.884949 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877307    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 10 22:55:37.884949 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877314    2563 flags.go:64] FLAG: --pod-manifest-path=""
Jul 10 22:55:37.884949 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877316    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 10 22:55:37.884949 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877319    2563 flags.go:64] FLAG: --pods-per-core="0"
Jul 10 22:55:37.884949 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877322    2563 flags.go:64] FLAG: --port="10250"
Jul 10 22:55:37.884949 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877325    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 10 22:55:37.884949 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877328    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00665a85c00697683"
Jul 10 22:55:37.884949 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877331    2563 flags.go:64] FLAG: --qos-reserved=""
Jul 10 22:55:37.884949 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877334    2563 flags.go:64] FLAG: --read-only-port="10255"
Jul 10 22:55:37.884949 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877336    2563 flags.go:64] FLAG: --register-node="true"
Jul 10 22:55:37.884949 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877339    2563 flags.go:64] FLAG: --register-with-taints=""
Jul 10 22:55:37.884949 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877343    2563 flags.go:64] FLAG: --registry-burst="10"
Jul 10 22:55:37.884949 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877346    2563 flags.go:64] FLAG: --registry-qps="5"
Jul 10 22:55:37.884949 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877350    2563 flags.go:64] FLAG: --reserved-cpus=""
Jul 10 22:55:37.884949 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877353    2563 flags.go:64] FLAG: --reserved-memory=""
Jul 10 22:55:37.884949 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877357    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 10 22:55:37.884949 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877360    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 10 22:55:37.884949 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877363    2563 flags.go:64] FLAG: --rotate-certificates="false"
Jul 10 22:55:37.884949 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877368    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 10 22:55:37.884949 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877371    2563 flags.go:64] FLAG: --runonce="false"
Jul 10 22:55:37.884949 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877374    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 10 22:55:37.885593 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877377    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 10 22:55:37.885593 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877380    2563 flags.go:64] FLAG: --seccomp-default="false"
Jul 10 22:55:37.885593 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877383    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 10 22:55:37.885593 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877399    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 10 22:55:37.885593 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877402    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 10 22:55:37.885593 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877405    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 10 22:55:37.885593 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877408    2563 flags.go:64] FLAG: --storage-driver-password="root"
Jul 10 22:55:37.885593 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877411    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 10 22:55:37.885593 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877414    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 10 22:55:37.885593 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877417    2563 flags.go:64] FLAG: --storage-driver-user="root"
Jul 10 22:55:37.885593 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877419    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 10 22:55:37.885593 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877423    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 10 22:55:37.885593 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877432    2563 flags.go:64] FLAG: --system-cgroups=""
Jul 10 22:55:37.885593 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877435    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 10 22:55:37.885593 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877440    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 10 22:55:37.885593 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877443    2563 flags.go:64] FLAG: --tls-cert-file=""
Jul 10 22:55:37.885593 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877446    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 10 22:55:37.885593 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877453    2563 flags.go:64] FLAG: --tls-min-version=""
Jul 10 22:55:37.885593 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877456    2563 flags.go:64] FLAG: --tls-private-key-file=""
Jul 10 22:55:37.885593 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877459    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 10 22:55:37.885593 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877461    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 10 22:55:37.885593 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877464    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 10 22:55:37.885593 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877467    2563 flags.go:64] FLAG: --v="2"
Jul 10 22:55:37.885593 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877471    2563 flags.go:64] FLAG: --version="false"
Jul 10 22:55:37.885593 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877478    2563 flags.go:64] FLAG: --vmodule=""
Jul 10 22:55:37.886232 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877482    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 10 22:55:37.886232 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.877485    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 10 22:55:37.886232 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877624    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 22:55:37.886232 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877629    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 22:55:37.886232 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877633    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 22:55:37.886232 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877637    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 22:55:37.886232 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877641    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 22:55:37.886232 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877643    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 22:55:37.886232 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877646    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 22:55:37.886232 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877649    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 22:55:37.886232 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877651    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 22:55:37.886232 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877654    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 22:55:37.886232 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877656    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 22:55:37.886232 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877659    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 22:55:37.886232 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877661    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 22:55:37.886232 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877664    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 22:55:37.886232 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877666    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 22:55:37.886232 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877669    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 22:55:37.886232 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877672    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 22:55:37.886232 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877674    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 22:55:37.887021 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877677    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 22:55:37.887021 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877679    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 22:55:37.887021 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877682    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 22:55:37.887021 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877685    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 22:55:37.887021 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877687    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 22:55:37.887021 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877690    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 22:55:37.887021 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877694    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 22:55:37.887021 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877697    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 22:55:37.887021 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877701    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 22:55:37.887021 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877703    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 22:55:37.887021 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877706    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 22:55:37.887021 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877711    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 22:55:37.887021 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877714    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 22:55:37.887021 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877717    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 22:55:37.887021 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877720    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 22:55:37.887021 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877722    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 22:55:37.887021 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877727    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 22:55:37.887021 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877729    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 22:55:37.887021 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877732    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 22:55:37.887674 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877734    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 22:55:37.887674 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877737    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 22:55:37.887674 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877739    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 22:55:37.887674 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877742    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 22:55:37.887674 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877744    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 22:55:37.887674 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877746    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 22:55:37.887674 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877749    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 22:55:37.887674 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877751    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 22:55:37.887674 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877754    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 22:55:37.887674 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877756    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 22:55:37.887674 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877759    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 22:55:37.887674 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877761    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 22:55:37.887674 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877764    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 22:55:37.887674 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877766    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 22:55:37.887674 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877769    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 22:55:37.887674 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877772    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 22:55:37.887674 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877774    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 22:55:37.887674 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877776    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 22:55:37.887674 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877779    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 22:55:37.887674 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877781    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 22:55:37.888215 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877784    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 22:55:37.888215 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877786    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 22:55:37.888215 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877788    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 22:55:37.888215 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877791    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 22:55:37.888215 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877794    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 22:55:37.888215 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877797    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 22:55:37.888215 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877800    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 22:55:37.888215 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877802    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 22:55:37.888215 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877805    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 22:55:37.888215 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877809    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 22:55:37.888215 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877811    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 22:55:37.888215 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877814    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 22:55:37.888215 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877816    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 22:55:37.888215 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877818    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 22:55:37.888215 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877821    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 22:55:37.888215 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877823    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 22:55:37.888215 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877826    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 22:55:37.888215 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877828    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 22:55:37.888215 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877830    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 22:55:37.888215 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877833    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 22:55:37.888718 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877835    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 22:55:37.888718 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877838    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 22:55:37.888718 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877840    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 22:55:37.888718 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877843    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 22:55:37.888718 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877846    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 22:55:37.888718 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877848    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 22:55:37.888718 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877850    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 22:55:37.888718 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877853    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 22:55:37.888718 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877856    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 22:55:37.888718 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877858    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 22:55:37.888718 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877861    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 22:55:37.888718 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877864    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 22:55:37.888718 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877866    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 22:55:37.888718 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877869    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 22:55:37.888718 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877871    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 22:55:37.888718 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877874    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 22:55:37.888718 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877877    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 22:55:37.888718 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877880    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 22:55:37.888718 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877882    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 22:55:37.889173 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877885    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 22:55:37.889173 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877887    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 22:55:37.889173 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.877891    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 22:55:37.889173 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.878415    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 22:55:37.889173 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.885329    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 10 22:55:37.889173 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.885357    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 10 22:55:37.889173 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.885372    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 10 22:55:37.889173 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.885378    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 10 22:55:37.889173 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885457    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 22:55:37.889173 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885462    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 22:55:37.889173 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885466    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 22:55:37.889173 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885469    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 22:55:37.889173 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885472    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 22:55:37.889173 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885475    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 22:55:37.889173 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885478    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 22:55:37.889594 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885480    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 22:55:37.889594 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885483    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 22:55:37.889594 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885486    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 22:55:37.889594 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885489    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 22:55:37.889594 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885491    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 22:55:37.889594 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885494    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 22:55:37.889594 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885496    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 22:55:37.889594 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885501    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 22:55:37.889594 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885506    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 22:55:37.889594 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885509    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 22:55:37.889594 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885512    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 22:55:37.889594 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885515    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 22:55:37.889594 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885518    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 22:55:37.889594 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885521    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 22:55:37.889594 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885524    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 22:55:37.889594 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885526    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 22:55:37.889594 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885529    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 22:55:37.889594 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885532    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 22:55:37.889594 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885534    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 22:55:37.889594 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885537    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 22:55:37.890075 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885539    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 22:55:37.890075 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885542    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 22:55:37.890075 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885552    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 22:55:37.890075 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885555    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 22:55:37.890075 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885558    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 22:55:37.890075 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885560    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 22:55:37.890075 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885563    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 22:55:37.890075 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885565    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 22:55:37.890075 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885568    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 22:55:37.890075 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885571    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 22:55:37.890075 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885573    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 22:55:37.890075 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885576    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 22:55:37.890075 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885578    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 22:55:37.890075 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885582    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 22:55:37.890075 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885584    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 22:55:37.890075 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885587    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 22:55:37.890075 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885589    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 22:55:37.890075 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885592    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 22:55:37.890075 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885594    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 22:55:37.890075 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885597    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 22:55:37.890629 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885599    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 22:55:37.890629 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885602    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 22:55:37.890629 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885605    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 22:55:37.890629 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885607    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 22:55:37.890629 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885610    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 22:55:37.890629 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885613    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 22:55:37.890629 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885615    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 22:55:37.890629 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885618    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 22:55:37.890629 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885620    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 22:55:37.890629 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885622    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 22:55:37.890629 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885625    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 22:55:37.890629 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885644    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 22:55:37.890629 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885647    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 22:55:37.890629 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885651    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 22:55:37.890629 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885655    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 22:55:37.890629 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885663    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 22:55:37.890629 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885667    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 22:55:37.890629 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885669    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 22:55:37.890629 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885672    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 22:55:37.890629 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885675    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 22:55:37.891177 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885677    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 22:55:37.891177 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885680    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 22:55:37.891177 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885683    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 22:55:37.891177 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885685    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 22:55:37.891177 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885687    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 22:55:37.891177 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885690    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 22:55:37.891177 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885692    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 22:55:37.891177 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885696    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 22:55:37.891177 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885699    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 22:55:37.891177 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885702    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 22:55:37.891177 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885704    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 22:55:37.891177 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885707    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 22:55:37.891177 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885709    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 22:55:37.891177 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885711    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 22:55:37.891177 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885715    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 22:55:37.891177 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885719    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 22:55:37.891177 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885722    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 22:55:37.891177 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885725    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 22:55:37.891177 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885727    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 22:55:37.891783 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885730    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 22:55:37.891783 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885732    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 22:55:37.891783 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885735    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 22:55:37.891783 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885737    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 22:55:37.891783 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885739    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 22:55:37.891783 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885742    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 22:55:37.891783 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885744    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 22:55:37.891783 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885748    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 22:55:37.891783 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885750    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 22:55:37.891783 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885759    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 22:55:37.891783 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885762    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 22:55:37.891783 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885765    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 22:55:37.891783 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885767    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 22:55:37.891783 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.885772    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 22:55:37.891783 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885926    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 22:55:37.892233 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885931    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 22:55:37.892233 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885934    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 22:55:37.892233 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885937    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 22:55:37.892233 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885940    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 22:55:37.892233 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885943    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 22:55:37.892233 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885946    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 22:55:37.892233 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885948    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 22:55:37.892233 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885951    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 22:55:37.892233 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885954    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 22:55:37.892233 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885956    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 22:55:37.892233 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885959    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 22:55:37.892233 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885961    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 22:55:37.892233 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885964    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 22:55:37.892233 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885966    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 22:55:37.892233 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885968    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 22:55:37.892233 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885971    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 22:55:37.892233 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885973    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 22:55:37.892233 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885976    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 22:55:37.892233 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885978    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 22:55:37.892233 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885980    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 22:55:37.893040 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885983    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 22:55:37.893040 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885985    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 22:55:37.893040 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885988    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 22:55:37.893040 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885990    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 22:55:37.893040 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885993    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 22:55:37.893040 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885996    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 22:55:37.893040 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.885998    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 22:55:37.893040 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886006    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 22:55:37.893040 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886009    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 22:55:37.893040 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886011    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 22:55:37.893040 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886014    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 22:55:37.893040 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886016    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 22:55:37.893040 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886018    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 22:55:37.893040 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886021    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 22:55:37.893040 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886023    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 22:55:37.893040 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886026    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 22:55:37.893040 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886028    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 22:55:37.893040 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886030    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 22:55:37.893040 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886032    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 22:55:37.893770 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886036    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 22:55:37.893770 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886040    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 22:55:37.893770 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886043    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 22:55:37.893770 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886047    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 22:55:37.893770 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886050    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 22:55:37.893770 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886053    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 22:55:37.893770 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886055    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 22:55:37.893770 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886058    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 22:55:37.893770 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886060    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 22:55:37.893770 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886063    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 22:55:37.893770 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886066    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 22:55:37.893770 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886069    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 22:55:37.893770 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886071    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 22:55:37.893770 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886073    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 22:55:37.893770 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886076    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 22:55:37.893770 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886223    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 22:55:37.893770 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886230    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 22:55:37.893770 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886233    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 22:55:37.893770 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886237    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 22:55:37.894293 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886239    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 22:55:37.894293 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.886242    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 22:55:37.894293 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.887812    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 22:55:37.894293 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.887955    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 22:55:37.894293 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.887961    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 22:55:37.894293 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.887964    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 22:55:37.894293 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.887968    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 22:55:37.894293 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.887972    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 22:55:37.894293 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.887976    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 22:55:37.894293 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.887979    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 22:55:37.894293 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.887982    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 22:55:37.894293 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.887985    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 22:55:37.894293 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.887987    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 22:55:37.894293 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.887990    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 22:55:37.894293 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.887993    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 22:55:37.894293 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.887996    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 22:55:37.894293 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.887999    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 22:55:37.894293 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.888001    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 22:55:37.894293 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.888004    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 22:55:37.894293 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.888007    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 22:55:37.895051 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.888010    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 22:55:37.895051 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.888013    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 22:55:37.895051 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.888015    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 22:55:37.895051 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.888017    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 22:55:37.895051 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.888020    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 22:55:37.895051 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.888022    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 22:55:37.895051 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.888025    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 22:55:37.895051 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.888027    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 22:55:37.895051 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.888031    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 22:55:37.895051 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.888033    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 22:55:37.895051 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.888036    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 22:55:37.895051 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.888039    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 22:55:37.895051 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.888041    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 22:55:37.895051 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.888043    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 22:55:37.895051 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.888046    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 22:55:37.895051 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.888053    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 22:55:37.895051 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.888058    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 22:55:37.895051 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.888063    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 22:55:37.895051 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.888067    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 22:55:37.895562 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:37.888070    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 22:55:37.895562 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.888076    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 22:55:37.895562 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.888221    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 10 22:55:37.895562 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.890245    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 10 22:55:37.895562 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.891453    2563 server.go:1019] "Starting client certificate rotation"
Jul 10 22:55:37.895562 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.891576    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 10 22:55:37.895562 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.891623    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 10 22:55:37.895562 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.893556    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 10 22:55:37.895562 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.894796    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 10 22:55:37.905450 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.905424    2563 log.go:25] "Validated CRI v1 runtime API"
Jul 10 22:55:37.907779 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.907760    2563 log.go:25] "Validated CRI v1 image API"
Jul 10 22:55:37.908743 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.908726    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 10 22:55:37.911348 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.911327    2563 fs.go:135] Filesystem UUIDs: map[31c32d6e-dcc1-4a6e-8b02-d41dcb5cdd8a:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 dd75ff19-6242-49dc-be86-58e1050872e2:/dev/nvme0n1p4]
Jul 10 22:55:37.911433 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.911346    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 10 22:55:37.917026 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.916883    2563 manager.go:216] Machine: {Timestamp:2026-07-10 22:55:37.916332898 +0000 UTC m=+0.267042122 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104272 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26166126652409695d31da4eaf28d6 SystemUUID:ec261661-2665-2409-695d-31da4eaf28d6 BootID:392026c2-9cab-4886-9f59-11447d825f3d Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:34:ac:9b:ae:5f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:34:ac:9b:ae:5f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:62:9a:7d:48:b0:d2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 10 22:55:37.917026 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.917008    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 10 22:55:37.918672 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.917232    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 10 22:55:37.919346 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.919327    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 10 22:55:37.921670 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.921639    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 10 22:55:37.921842 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.921679    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-239.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 10 22:55:37.922369 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.922355    2563 topology_manager.go:138] "Creating topology manager with none policy"
Jul 10 22:55:37.922438 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.922376    2563 container_manager_linux.go:309] "Creating device plugin manager"
Jul 10 22:55:37.922438 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.922404    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 10 22:55:37.923084 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.923072    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 10 22:55:37.924446 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.924431    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 10 22:55:37.928760 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.928740    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 10 22:55:37.928924 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.928911    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 10 22:55:37.928981 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.928955    2563 kubelet.go:486] "Attempting to sync node with API server"
Jul 10 22:55:37.928981 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.928976    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 10 22:55:37.929061 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.928997    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 10 22:55:37.929061 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.929013    2563 kubelet.go:398] "Adding apiserver pod source"
Jul 10 22:55:37.929061 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.929026    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 10 22:55:37.929452 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.929437    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 10 22:55:37.929527 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.929461    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 10 22:55:37.931927 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.931908    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 10 22:55:37.933767 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.933753    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 10 22:55:37.933825 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.933784    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 10 22:55:37.933954 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.933943    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 10 22:55:37.933985 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.933962    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 10 22:55:37.933985 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.933970    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 10 22:55:37.933985 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.933977    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 10 22:55:37.933985 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.933985    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 10 22:55:37.934094 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.933991    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 10 22:55:37.934410 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.934401    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 10 22:55:37.934410 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.934410    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 10 22:55:37.934464 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.934418    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 10 22:55:37.934464 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.934424    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 10 22:55:37.934464 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.934436    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 10 22:55:37.934464 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.934445    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 10 22:55:37.935772 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.935666    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 10 22:55:37.935819 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.935780    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 10 22:55:37.939182 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.939166    2563 server.go:1268] "Started kubelet"
Jul 10 22:55:37.939317 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.939249    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 10 22:55:37.939420 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.939299    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 10 22:55:37.939487 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.939473    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 10 22:55:37.940250 ip-10-0-141-239 systemd[1]: Started Kubernetes Kubelet.
Jul 10 22:55:37.940898 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.940797    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 10 22:55:37.942209 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.942194    2563 server.go:310] "Adding debug handlers to kubelet server"
Jul 10 22:55:37.942824 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.942803    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-239.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 10 22:55:37.942895 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:37.942805    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 10 22:55:37.942895 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:37.942808    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-239.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 10 22:55:37.944514 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:37.944489    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 10 22:55:37.944938 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:37.944179    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-239.ec2.internal.18c10fa72326d99f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-239.ec2.internal,UID:ip-10-0-141-239.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-239.ec2.internal,},FirstTimestamp:2026-07-10 22:55:37.939130783 +0000 UTC m=+0.289840005,LastTimestamp:2026-07-10 22:55:37.939130783 +0000 UTC m=+0.289840005,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-239.ec2.internal,}"
Jul 10 22:55:37.946556 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.946541    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 10 22:55:37.946606 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.946555    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 10 22:55:37.947111 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:37.946913    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-239.ec2.internal\" not found"
Jul 10 22:55:37.947276 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.947236    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 10 22:55:37.947365 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.947245    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 10 22:55:37.947365 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.947298    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 10 22:55:37.947539 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.947464    2563 reconstruct.go:97] "Volume reconstruction finished"
Jul 10 22:55:37.947539 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.947474    2563 reconciler.go:29] "Reconciler: start to sync state"
Jul 10 22:55:37.947600 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.947560    2563 factory.go:55] Registering systemd factory
Jul 10 22:55:37.947600 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.947575    2563 factory.go:223] Registration of the systemd container factory successfully
Jul 10 22:55:37.949035 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.948015    2563 factory.go:153] Registering CRI-O factory
Jul 10 22:55:37.949035 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.948032    2563 factory.go:223] Registration of the crio container factory successfully
Jul 10 22:55:37.949035 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.948084    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 10 22:55:37.949035 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.948105    2563 factory.go:103] Registering Raw factory
Jul 10 22:55:37.949035 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.948120    2563 manager.go:1196] Started watching for new ooms in manager
Jul 10 22:55:37.949035 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.948769    2563 manager.go:349] Starting recovery of all containers
Jul 10 22:55:37.959571 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.959376    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 10 22:55:37.959830 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:37.959799    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-239.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 10 22:55:37.959934 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:37.959841    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 10 22:55:37.961634 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.961615    2563 manager.go:354] Recovery completed
Jul 10 22:55:37.964154 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.964123    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2wl8h"
Jul 10 22:55:37.965960 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.965948    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 22:55:37.967967 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.967948    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-239.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 22:55:37.968055 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.967979    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-239.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 22:55:37.968055 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.967990    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-239.ec2.internal" event="NodeHasSufficientPID"
Jul 10 22:55:37.968546 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.968525    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 10 22:55:37.968546 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.968538    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 10 22:55:37.968681 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.968553    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 10 22:55:37.968681 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.968582    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 22:55:37.969254 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.969234    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-239.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 22:55:37.969308 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.969269    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-239.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 22:55:37.969308 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.969282    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-239.ec2.internal" event="NodeHasSufficientPID"
Jul 10 22:55:37.970703 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.970691    2563 policy_none.go:49] "None policy: Start"
Jul 10 22:55:37.970748 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.970707    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 10 22:55:37.970748 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.970717    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 10 22:55:37.971645 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.971631    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2wl8h"
Jul 10 22:55:37.972676 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:37.972663    2563 policy_none.go:47] "Start"
Jul 10 22:55:37.976127 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:37.976055    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-239.ec2.internal.18c10fa724ded587  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-239.ec2.internal,UID:ip-10-0-141-239.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-239.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-239.ec2.internal,},FirstTimestamp:2026-07-10 22:55:37.967965575 +0000 UTC m=+0.318674802,LastTimestamp:2026-07-10 22:55:37.967965575 +0000 UTC m=+0.318674802,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-239.ec2.internal,}"
Jul 10 22:55:38.017276 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.017261    2563 manager.go:337] "Starting Device Plugin manager"
Jul 10 22:55:38.017359 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:38.017303    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 10 22:55:38.017359 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.017313    2563 server.go:87] "Starting device plugin registration server"
Jul 10 22:55:38.017618 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.017607    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 10 22:55:38.017663 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.017620    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 10 22:55:38.017727 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.017713    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 10 22:55:38.017800 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.017787    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 10 22:55:38.017851 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.017801    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 10 22:55:38.018488 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:38.018454    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 10 22:55:38.018563 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:38.018515    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-239.ec2.internal\" not found"
Jul 10 22:55:38.085603 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.085567    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 10 22:55:38.087876 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.085618    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 10 22:55:38.087876 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.085644    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 10 22:55:38.087876 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:38.085694    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 10 22:55:38.088040 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.087987    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 22:55:38.117953 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.117906    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 22:55:38.118978 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.118962    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-239.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 22:55:38.119102 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.118993    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-239.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 22:55:38.119102 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.119005    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-239.ec2.internal" event="NodeHasSufficientPID"
Jul 10 22:55:38.119102 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.119027    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-239.ec2.internal"
Jul 10 22:55:38.128674 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.128648    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-239.ec2.internal"
Jul 10 22:55:38.128762 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:38.128679    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-239.ec2.internal\": node \"ip-10-0-141-239.ec2.internal\" not found"
Jul 10 22:55:38.152886 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:38.152855    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-239.ec2.internal\" not found"
Jul 10 22:55:38.186183 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.186148    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-239.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-239.ec2.internal"]
Jul 10 22:55:38.186267 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.186225    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 22:55:38.187212 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.187190    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-239.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 22:55:38.187319 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.187224    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-239.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 22:55:38.187319 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.187233    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-239.ec2.internal" event="NodeHasSufficientPID"
Jul 10 22:55:38.189650 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.189635    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 22:55:38.189786 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.189772    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-239.ec2.internal"
Jul 10 22:55:38.189829 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.189802    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 22:55:38.190417 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.190383    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-239.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 22:55:38.190496 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.190405    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-239.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 22:55:38.190496 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.190445    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-239.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 22:55:38.190496 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.190454    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-239.ec2.internal" event="NodeHasSufficientPID"
Jul 10 22:55:38.190582 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.190428    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-239.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 22:55:38.190582 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.190508    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-239.ec2.internal" event="NodeHasSufficientPID"
Jul 10 22:55:38.192797 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.192777    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-239.ec2.internal"
Jul 10 22:55:38.192859 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.192818    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 22:55:38.193548 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.193531    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-239.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 22:55:38.193614 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.193566    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-239.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 22:55:38.193614 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.193578    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-239.ec2.internal" event="NodeHasSufficientPID"
Jul 10 22:55:38.217272 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:38.217240    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-239.ec2.internal\" not found" node="ip-10-0-141-239.ec2.internal"
Jul 10 22:55:38.221873 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:38.221848    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-239.ec2.internal\" not found" node="ip-10-0-141-239.ec2.internal"
Jul 10 22:55:38.248338 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.248306    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/98bd729d123769a48c9d58a5f9915bea-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-239.ec2.internal\" (UID: \"98bd729d123769a48c9d58a5f9915bea\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-239.ec2.internal"
Jul 10 22:55:38.248338 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.248337    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98bd729d123769a48c9d58a5f9915bea-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-239.ec2.internal\" (UID: \"98bd729d123769a48c9d58a5f9915bea\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-239.ec2.internal"
Jul 10 22:55:38.248531 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.248355    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6c99c1f4b605d867ddb0d2d31d3a10b5-config\") pod \"kube-apiserver-proxy-ip-10-0-141-239.ec2.internal\" (UID: \"6c99c1f4b605d867ddb0d2d31d3a10b5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-239.ec2.internal"
Jul 10 22:55:38.253404 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:38.253368    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-239.ec2.internal\" not found"
Jul 10 22:55:38.348580 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.348541    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/98bd729d123769a48c9d58a5f9915bea-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-239.ec2.internal\" (UID: \"98bd729d123769a48c9d58a5f9915bea\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-239.ec2.internal"
Jul 10 22:55:38.348580 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.348567    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/98bd729d123769a48c9d58a5f9915bea-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-239.ec2.internal\" (UID: \"98bd729d123769a48c9d58a5f9915bea\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-239.ec2.internal"
Jul 10 22:55:38.348765 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.348609    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98bd729d123769a48c9d58a5f9915bea-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-239.ec2.internal\" (UID: \"98bd729d123769a48c9d58a5f9915bea\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-239.ec2.internal"
Jul 10 22:55:38.348765 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.348636    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6c99c1f4b605d867ddb0d2d31d3a10b5-config\") pod \"kube-apiserver-proxy-ip-10-0-141-239.ec2.internal\" (UID: \"6c99c1f4b605d867ddb0d2d31d3a10b5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-239.ec2.internal"
Jul 10 22:55:38.348765 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.348656    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98bd729d123769a48c9d58a5f9915bea-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-239.ec2.internal\" (UID: \"98bd729d123769a48c9d58a5f9915bea\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-239.ec2.internal"
Jul 10 22:55:38.348765 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.348697    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6c99c1f4b605d867ddb0d2d31d3a10b5-config\") pod \"kube-apiserver-proxy-ip-10-0-141-239.ec2.internal\" (UID: \"6c99c1f4b605d867ddb0d2d31d3a10b5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-239.ec2.internal"
Jul 10 22:55:38.353657 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:38.353631    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-239.ec2.internal\" not found"
Jul 10 22:55:38.454594 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:38.454509    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-239.ec2.internal\" not found"
Jul 10 22:55:38.519742 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.519708    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-239.ec2.internal"
Jul 10 22:55:38.524886 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:38.524757    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-239.ec2.internal"
Jul 10 22:55:38.555046 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:38.555002    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-239.ec2.internal\" not found"
Jul 10 22:55:38.655691 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:38.655636    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-239.ec2.internal\" not found"
Jul 10 22:55:40.220328 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:40.220242    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-239.ec2.internal\" not found"
Jul 10 22:55:40.289504 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.289467    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 22:55:40.311465 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.311430    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-239.ec2.internal"
Jul 10 22:55:40.324196 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.324175    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 10 22:55:40.325655 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.325639    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-239.ec2.internal"
Jul 10 22:55:40.332429 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.332411    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 10 22:55:40.355541 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.355508    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 10 22:55:40.356132 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.355683    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 10 22:55:40.356132 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.355719    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 10 22:55:40.393888 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.393854    2563 apiserver.go:52] "Watching apiserver"
Jul 10 22:55:40.411607 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.411574    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 10 22:55:40.411792 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.411779    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 10 22:55:40.412754 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.412731    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-7zb67","kube-system/konnectivity-agent-fwk45","openshift-cluster-node-tuning-operator/tuned-w2wjh","openshift-dns/node-resolver-kd62p","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-239.ec2.internal","openshift-multus/multus-additional-cni-plugins-xpkzr","openshift-multus/network-metrics-daemon-w9kxw","openshift-network-diagnostics/network-check-target-8nt9g","kube-system/kube-apiserver-proxy-ip-10-0-141-239.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks","openshift-image-registry/node-ca-298xj","openshift-multus/multus-rdxmp","openshift-network-operator/iptables-alerter-hvkfr","openshift-ovn-kubernetes/ovnkube-node-qj86t"]
Jul 10 22:55:40.415396 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.415378    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:55:40.415462 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:40.415448    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9kxw" podUID="9829a27f-3dbd-44c7-b278-f26beb69f82a"
Jul 10 22:55:40.417638 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.417619    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-298xj"
Jul 10 22:55:40.419997 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.419975    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.420330 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.420299    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 10 22:55:40.420468 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.420335    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 10 22:55:40.420468 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.420302    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 10 22:55:40.420568 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.420508    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pbg2w\""
Jul 10 22:55:40.422176 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.422157    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kd62p"
Jul 10 22:55:40.422894 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.422873    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 10 22:55:40.423001 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.422874    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 10 22:55:40.423238 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.423225    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hhmqg\""
Jul 10 22:55:40.424074 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.424060    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 10 22:55:40.424136 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.424106    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-8gszv\""
Jul 10 22:55:40.424289 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.424270    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 10 22:55:40.424563 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.424548    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.425138 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.425122    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 10 22:55:40.426321 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.426302    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 10 22:55:40.426816 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.426798    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 10 22:55:40.426898 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.426843    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 10 22:55:40.426898 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.426893    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xpkzr"
Jul 10 22:55:40.427601 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.427295    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 10 22:55:40.428048 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.428024    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6tbfw\""
Jul 10 22:55:40.428136 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.428082    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 10 22:55:40.428746 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.428470    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 10 22:55:40.429634 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.429485    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 10 22:55:40.429634 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.429557    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 10 22:55:40.429838 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.429823    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 10 22:55:40.429969 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.429927    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xhgt2\""
Jul 10 22:55:40.430149 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.430132    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:55:40.430241 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:40.430221    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nt9g" podUID="3f0e1c5a-a319-437d-9aef-8b3c4150a3b1"
Jul 10 22:55:40.430302 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.430244    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 10 22:55:40.430352 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.430305    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 10 22:55:40.433529 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.433510    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hvkfr"
Jul 10 22:55:40.435561 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.435540    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 10 22:55:40.435763 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.435748    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 10 22:55:40.435849 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.435803    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 10 22:55:40.435849 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.435833    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vb4r4\""
Jul 10 22:55:40.436066 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.436051    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fwk45"
Jul 10 22:55:40.438180 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.438162    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks"
Jul 10 22:55:40.438280 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.438212    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-km6fw\""
Jul 10 22:55:40.438344 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.438282    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 10 22:55:40.438436 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.438385    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 10 22:55:40.440020 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.439980    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 22:50:39 +0000 UTC" deadline="2028-02-01 02:53:30.554843413 +0000 UTC"
Jul 10 22:55:40.440071 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.440020    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13683h57m50.114826155s"
Jul 10 22:55:40.440219 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.440207    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 10 22:55:40.440317 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.440301    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 10 22:55:40.440385 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.440337    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-m7gtr\""
Jul 10 22:55:40.440385 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.440349    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 10 22:55:40.440576 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.440562    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:55:40.440644 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:40.440627    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zb67" podUID="3734910e-9249-4060-afc4-55140bcb9911"
Jul 10 22:55:40.442819 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.442804    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.444696 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.444677    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mxr9s\""
Jul 10 22:55:40.444794 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.444756    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 10 22:55:40.450437 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.450420    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-f7tmk"
Jul 10 22:55:40.458940 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.458916    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-f7tmk"
Jul 10 22:55:40.512600 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.512575    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 10 22:55:40.516263 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.516239    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-host\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.516409 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.516273    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e17cacf8-0106-484f-a226-2f584a8fc5a5-host\") pod \"node-ca-298xj\" (UID: \"e17cacf8-0106-484f-a226-2f584a8fc5a5\") " pod="openshift-image-registry/node-ca-298xj"
Jul 10 22:55:40.516409 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.516307    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-lib-modules\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.516409 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.516344    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-log-socket\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.516409 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.516387    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-host-run-ovn-kubernetes\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.516554 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.516418    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfbs6\" (UniqueName: \"kubernetes.io/projected/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-kube-api-access-jfbs6\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.516554 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.516449    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-multus-socket-dir-parent\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.516904 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.516882    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4szdb\" (UniqueName: \"kubernetes.io/projected/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-kube-api-access-4szdb\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.516939 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.516912    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/50a4a720-01af-4105-9de4-260ee26283a4-etc-tuned\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.516939 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.516931    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54d84e33-5ba4-4b6c-8c8f-4ddaac24eb03-tmp-dir\") pod \"node-resolver-kd62p\" (UID: \"54d84e33-5ba4-4b6c-8c8f-4ddaac24eb03\") " pod="openshift-dns/node-resolver-kd62p"
Jul 10 22:55:40.517000 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.516946    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cdbc917d-2667-438e-928b-326933a924d8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xpkzr\" (UID: \"cdbc917d-2667-438e-928b-326933a924d8\") " pod="openshift-multus/multus-additional-cni-plugins-xpkzr"
Jul 10 22:55:40.517000 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.516961    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cdbc917d-2667-438e-928b-326933a924d8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xpkzr\" (UID: \"cdbc917d-2667-438e-928b-326933a924d8\") " pod="openshift-multus/multus-additional-cni-plugins-xpkzr"
Jul 10 22:55:40.517000 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.516981    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-etc-sysconfig\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.517000 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.516995    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-etc-systemd\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.517135 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517045    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-ovn-node-metrics-cert\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.517135 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517107    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e17cacf8-0106-484f-a226-2f584a8fc5a5-serviceca\") pod \"node-ca-298xj\" (UID: \"e17cacf8-0106-484f-a226-2f584a8fc5a5\") " pod="openshift-image-registry/node-ca-298xj"
Jul 10 22:55:40.517204 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517146    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xmms\" (UniqueName: \"kubernetes.io/projected/e17cacf8-0106-484f-a226-2f584a8fc5a5-kube-api-access-7xmms\") pod \"node-ca-298xj\" (UID: \"e17cacf8-0106-484f-a226-2f584a8fc5a5\") " pod="openshift-image-registry/node-ca-298xj"
Jul 10 22:55:40.517204 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517186    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-run\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.517274 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517209    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-system-cni-dir\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.517274 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517232    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-host-run-netns\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.517274 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517253    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-host-var-lib-cni-bin\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.517397 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517273    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-multus-conf-dir\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.517397 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517304    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0ee6c937-5bad-4104-916a-71224add52b5-agent-certs\") pod \"konnectivity-agent-fwk45\" (UID: \"0ee6c937-5bad-4104-916a-71224add52b5\") " pod="kube-system/konnectivity-agent-fwk45"
Jul 10 22:55:40.517397 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517323    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-etc-sysctl-d\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.517397 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517337    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-host-cni-netd\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.517397 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517371    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-ovnkube-config\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.517598 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517395    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cdbc917d-2667-438e-928b-326933a924d8-cnibin\") pod \"multus-additional-cni-plugins-xpkzr\" (UID: \"cdbc917d-2667-438e-928b-326933a924d8\") " pod="openshift-multus/multus-additional-cni-plugins-xpkzr"
Jul 10 22:55:40.517598 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517422    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-hostroot\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.517598 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517437    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-etc-openvswitch\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.517598 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517467    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/257b0f09-db72-43e3-a3d5-a28a4fcdc3d6-socket-dir\") pod \"aws-ebs-csi-driver-node-xbkks\" (UID: \"257b0f09-db72-43e3-a3d5-a28a4fcdc3d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks"
Jul 10 22:55:40.517598 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517481    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/257b0f09-db72-43e3-a3d5-a28a4fcdc3d6-device-dir\") pod \"aws-ebs-csi-driver-node-xbkks\" (UID: \"257b0f09-db72-43e3-a3d5-a28a4fcdc3d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks"
Jul 10 22:55:40.517598 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517532    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjqs6\" (UniqueName: \"kubernetes.io/projected/9829a27f-3dbd-44c7-b278-f26beb69f82a-kube-api-access-gjqs6\") pod \"network-metrics-daemon-w9kxw\" (UID: \"9829a27f-3dbd-44c7-b278-f26beb69f82a\") " pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:55:40.517598 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517562    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-host-var-lib-cni-multus\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.517598 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517591    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/257b0f09-db72-43e3-a3d5-a28a4fcdc3d6-registration-dir\") pod \"aws-ebs-csi-driver-node-xbkks\" (UID: \"257b0f09-db72-43e3-a3d5-a28a4fcdc3d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks"
Jul 10 22:55:40.517961 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517614    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-var-lib-openvswitch\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.517961 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517637    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-run-ovn\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.517961 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517660    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-node-log\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.517961 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517694    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-host-cni-bin\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.517961 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517718    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cdbc917d-2667-438e-928b-326933a924d8-cni-binary-copy\") pod \"multus-additional-cni-plugins-xpkzr\" (UID: \"cdbc917d-2667-438e-928b-326933a924d8\") " pod="openshift-multus/multus-additional-cni-plugins-xpkzr"
Jul 10 22:55:40.517961 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517761    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-host-run-multus-certs\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.517961 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517805    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/257b0f09-db72-43e3-a3d5-a28a4fcdc3d6-etc-selinux\") pod \"aws-ebs-csi-driver-node-xbkks\" (UID: \"257b0f09-db72-43e3-a3d5-a28a4fcdc3d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks"
Jul 10 22:55:40.517961 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517831    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-etc-modprobe-d\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.517961 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517854    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5x9d\" (UniqueName: \"kubernetes.io/projected/54d84e33-5ba4-4b6c-8c8f-4ddaac24eb03-kube-api-access-t5x9d\") pod \"node-resolver-kd62p\" (UID: \"54d84e33-5ba4-4b6c-8c8f-4ddaac24eb03\") " pod="openshift-dns/node-resolver-kd62p"
Jul 10 22:55:40.517961 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517886    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-ovnkube-script-lib\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.517961 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517910    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3fe0a8af-e7c7-47cf-81d5-5e0c3c0ad4e2-iptables-alerter-script\") pod \"iptables-alerter-hvkfr\" (UID: \"3fe0a8af-e7c7-47cf-81d5-5e0c3c0ad4e2\") " pod="openshift-network-operator/iptables-alerter-hvkfr"
Jul 10 22:55:40.517961 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517941    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-cni-binary-copy\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.517961 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.517962    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-etc-sysctl-conf\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.518585 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.518005    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/50a4a720-01af-4105-9de4-260ee26283a4-tmp\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.518585 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.518028    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-etc-kubernetes\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.518585 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.518056    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-host-run-netns\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.518585 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.518084    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-cnibin\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.518585 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.518158    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-host-run-k8s-cni-cncf-io\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.518585 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.518182    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-multus-daemon-config\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.518585 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.518204    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8924\" (UniqueName: \"kubernetes.io/projected/3f0e1c5a-a319-437d-9aef-8b3c4150a3b1-kube-api-access-j8924\") pod \"network-check-target-8nt9g\" (UID: \"3f0e1c5a-a319-437d-9aef-8b3c4150a3b1\") " pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:55:40.518585 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.518248    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdnkc\" (UniqueName: \"kubernetes.io/projected/50a4a720-01af-4105-9de4-260ee26283a4-kube-api-access-zdnkc\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.518585 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.518310    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-host-kubelet\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.518585 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.518380    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-systemd-units\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.518585 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.518416    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.518585 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.518442    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrzl5\" (UniqueName: \"kubernetes.io/projected/3fe0a8af-e7c7-47cf-81d5-5e0c3c0ad4e2-kube-api-access-xrzl5\") pod \"iptables-alerter-hvkfr\" (UID: \"3fe0a8af-e7c7-47cf-81d5-5e0c3c0ad4e2\") " pod="openshift-network-operator/iptables-alerter-hvkfr"
Jul 10 22:55:40.518585 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.518458    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3734910e-9249-4060-afc4-55140bcb9911-original-pull-secret\") pod \"global-pull-secret-syncer-7zb67\" (UID: \"3734910e-9249-4060-afc4-55140bcb9911\") " pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:55:40.518585 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.518472    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/257b0f09-db72-43e3-a3d5-a28a4fcdc3d6-sys-fs\") pod \"aws-ebs-csi-driver-node-xbkks\" (UID: \"257b0f09-db72-43e3-a3d5-a28a4fcdc3d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks"
Jul 10 22:55:40.518585 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.518488    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cdbc917d-2667-438e-928b-326933a924d8-os-release\") pod \"multus-additional-cni-plugins-xpkzr\" (UID: \"cdbc917d-2667-438e-928b-326933a924d8\") " pod="openshift-multus/multus-additional-cni-plugins-xpkzr"
Jul 10 22:55:40.519201 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.518892    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3fe0a8af-e7c7-47cf-81d5-5e0c3c0ad4e2-host-slash\") pod \"iptables-alerter-hvkfr\" (UID: \"3fe0a8af-e7c7-47cf-81d5-5e0c3c0ad4e2\") " pod="openshift-network-operator/iptables-alerter-hvkfr"
Jul 10 22:55:40.519201 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.519003    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-multus-cni-dir\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.519201 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.519067    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-host-var-lib-kubelet\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.519201 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.519109    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs\") pod \"network-metrics-daemon-w9kxw\" (UID: \"9829a27f-3dbd-44c7-b278-f26beb69f82a\") " pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:55:40.519201 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.519145    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-var-lib-kubelet\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.519201 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.519178    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-run-openvswitch\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.519533 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.519212    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3734910e-9249-4060-afc4-55140bcb9911-dbus\") pod \"global-pull-secret-syncer-7zb67\" (UID: \"3734910e-9249-4060-afc4-55140bcb9911\") " pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:55:40.519533 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.519243    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-os-release\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.519533 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.519279    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/257b0f09-db72-43e3-a3d5-a28a4fcdc3d6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xbkks\" (UID: \"257b0f09-db72-43e3-a3d5-a28a4fcdc3d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks"
Jul 10 22:55:40.519533 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.519311    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-sys\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.519533 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.519347    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/54d84e33-5ba4-4b6c-8c8f-4ddaac24eb03-hosts-file\") pod \"node-resolver-kd62p\" (UID: \"54d84e33-5ba4-4b6c-8c8f-4ddaac24eb03\") " pod="openshift-dns/node-resolver-kd62p"
Jul 10 22:55:40.519533 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.519395    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-run-systemd\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.519533 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.519515    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btjmf\" (UniqueName: \"kubernetes.io/projected/cdbc917d-2667-438e-928b-326933a924d8-kube-api-access-btjmf\") pod \"multus-additional-cni-plugins-xpkzr\" (UID: \"cdbc917d-2667-438e-928b-326933a924d8\") " pod="openshift-multus/multus-additional-cni-plugins-xpkzr"
Jul 10 22:55:40.519880 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.519637    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0ee6c937-5bad-4104-916a-71224add52b5-konnectivity-ca\") pod \"konnectivity-agent-fwk45\" (UID: \"0ee6c937-5bad-4104-916a-71224add52b5\") " pod="kube-system/konnectivity-agent-fwk45"
Jul 10 22:55:40.519880 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.519699    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-host-slash\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.519880 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.519757    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-env-overrides\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.519880 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.519810    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cdbc917d-2667-438e-928b-326933a924d8-system-cni-dir\") pod \"multus-additional-cni-plugins-xpkzr\" (UID: \"cdbc917d-2667-438e-928b-326933a924d8\") " pod="openshift-multus/multus-additional-cni-plugins-xpkzr"
Jul 10 22:55:40.520040 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.519976    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cdbc917d-2667-438e-928b-326933a924d8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xpkzr\" (UID: \"cdbc917d-2667-438e-928b-326933a924d8\") " pod="openshift-multus/multus-additional-cni-plugins-xpkzr"
Jul 10 22:55:40.520040 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.520018    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3734910e-9249-4060-afc4-55140bcb9911-kubelet-config\") pod \"global-pull-secret-syncer-7zb67\" (UID: \"3734910e-9249-4060-afc4-55140bcb9911\") " pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:55:40.520130 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.520053    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krzx9\" (UniqueName: \"kubernetes.io/projected/257b0f09-db72-43e3-a3d5-a28a4fcdc3d6-kube-api-access-krzx9\") pod \"aws-ebs-csi-driver-node-xbkks\" (UID: \"257b0f09-db72-43e3-a3d5-a28a4fcdc3d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks"
Jul 10 22:55:40.520255 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.520238    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-etc-kubernetes\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.621499 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.621459    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8924\" (UniqueName: \"kubernetes.io/projected/3f0e1c5a-a319-437d-9aef-8b3c4150a3b1-kube-api-access-j8924\") pod \"network-check-target-8nt9g\" (UID: \"3f0e1c5a-a319-437d-9aef-8b3c4150a3b1\") " pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:55:40.621499 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.621502    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdnkc\" (UniqueName: \"kubernetes.io/projected/50a4a720-01af-4105-9de4-260ee26283a4-kube-api-access-zdnkc\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.621787 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.621525    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-host-kubelet\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.621787 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.621545    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-systemd-units\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.621787 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.621583    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.621787 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.621609    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrzl5\" (UniqueName: \"kubernetes.io/projected/3fe0a8af-e7c7-47cf-81d5-5e0c3c0ad4e2-kube-api-access-xrzl5\") pod \"iptables-alerter-hvkfr\" (UID: \"3fe0a8af-e7c7-47cf-81d5-5e0c3c0ad4e2\") " pod="openshift-network-operator/iptables-alerter-hvkfr"
Jul 10 22:55:40.621787 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.621663    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-host-kubelet\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.621787 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.621693    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-systemd-units\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.621787 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.621694    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.621787 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.621722    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3734910e-9249-4060-afc4-55140bcb9911-original-pull-secret\") pod \"global-pull-secret-syncer-7zb67\" (UID: \"3734910e-9249-4060-afc4-55140bcb9911\") " pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:55:40.621787 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.621757    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/257b0f09-db72-43e3-a3d5-a28a4fcdc3d6-sys-fs\") pod \"aws-ebs-csi-driver-node-xbkks\" (UID: \"257b0f09-db72-43e3-a3d5-a28a4fcdc3d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks"
Jul 10 22:55:40.621787 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.621781    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cdbc917d-2667-438e-928b-326933a924d8-os-release\") pod \"multus-additional-cni-plugins-xpkzr\" (UID: \"cdbc917d-2667-438e-928b-326933a924d8\") " pod="openshift-multus/multus-additional-cni-plugins-xpkzr"
Jul 10 22:55:40.622263 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.621798    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3fe0a8af-e7c7-47cf-81d5-5e0c3c0ad4e2-host-slash\") pod \"iptables-alerter-hvkfr\" (UID: \"3fe0a8af-e7c7-47cf-81d5-5e0c3c0ad4e2\") " pod="openshift-network-operator/iptables-alerter-hvkfr"
Jul 10 22:55:40.622263 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.621817    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-multus-cni-dir\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.622263 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:40.621828    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 22:55:40.622263 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.621838    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-host-var-lib-kubelet\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.622263 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.621876    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3fe0a8af-e7c7-47cf-81d5-5e0c3c0ad4e2-host-slash\") pod \"iptables-alerter-hvkfr\" (UID: \"3fe0a8af-e7c7-47cf-81d5-5e0c3c0ad4e2\") " pod="openshift-network-operator/iptables-alerter-hvkfr"
Jul 10 22:55:40.622263 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.621880    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-multus-cni-dir\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.622263 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.621880    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs\") pod \"network-metrics-daemon-w9kxw\" (UID: \"9829a27f-3dbd-44c7-b278-f26beb69f82a\") " pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:55:40.622263 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.621828    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/257b0f09-db72-43e3-a3d5-a28a4fcdc3d6-sys-fs\") pod \"aws-ebs-csi-driver-node-xbkks\" (UID: \"257b0f09-db72-43e3-a3d5-a28a4fcdc3d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks"
Jul 10 22:55:40.622263 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:40.621916    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3734910e-9249-4060-afc4-55140bcb9911-original-pull-secret podName:3734910e-9249-4060-afc4-55140bcb9911 nodeName:}" failed. No retries permitted until 2026-07-10 22:55:41.121878416 +0000 UTC m=+2.008614537 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3734910e-9249-4060-afc4-55140bcb9911-original-pull-secret") pod "global-pull-secret-syncer-7zb67" (UID: "3734910e-9249-4060-afc4-55140bcb9911") : object "kube-system"/"original-pull-secret" not registered
Jul 10 22:55:40.622263 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.621879    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cdbc917d-2667-438e-928b-326933a924d8-os-release\") pod \"multus-additional-cni-plugins-xpkzr\" (UID: \"cdbc917d-2667-438e-928b-326933a924d8\") " pod="openshift-multus/multus-additional-cni-plugins-xpkzr"
Jul 10 22:55:40.622263 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:40.621937    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 22:55:40.622263 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.621915    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-host-var-lib-kubelet\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.622263 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.621971    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-var-lib-kubelet\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.622263 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:40.621985    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs podName:9829a27f-3dbd-44c7-b278-f26beb69f82a nodeName:}" failed. No retries permitted until 2026-07-10 22:55:41.121967495 +0000 UTC m=+2.008703629 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs") pod "network-metrics-daemon-w9kxw" (UID: "9829a27f-3dbd-44c7-b278-f26beb69f82a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 22:55:40.622263 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622010    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-run-openvswitch\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.622263 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622026    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3734910e-9249-4060-afc4-55140bcb9911-dbus\") pod \"global-pull-secret-syncer-7zb67\" (UID: \"3734910e-9249-4060-afc4-55140bcb9911\") " pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:55:40.622263 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622048    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-os-release\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.622825 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622066    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/257b0f09-db72-43e3-a3d5-a28a4fcdc3d6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xbkks\" (UID: \"257b0f09-db72-43e3-a3d5-a28a4fcdc3d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks"
Jul 10 22:55:40.622825 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622082    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-sys\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.622825 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622097    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/54d84e33-5ba4-4b6c-8c8f-4ddaac24eb03-hosts-file\") pod \"node-resolver-kd62p\" (UID: \"54d84e33-5ba4-4b6c-8c8f-4ddaac24eb03\") " pod="openshift-dns/node-resolver-kd62p"
Jul 10 22:55:40.622825 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622120    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/257b0f09-db72-43e3-a3d5-a28a4fcdc3d6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xbkks\" (UID: \"257b0f09-db72-43e3-a3d5-a28a4fcdc3d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks"
Jul 10 22:55:40.622825 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622133    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-sys\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.622825 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622121    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-os-release\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.622825 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622153    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-run-systemd\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.622825 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622094    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-var-lib-kubelet\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.622825 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622163    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/54d84e33-5ba4-4b6c-8c8f-4ddaac24eb03-hosts-file\") pod \"node-resolver-kd62p\" (UID: \"54d84e33-5ba4-4b6c-8c8f-4ddaac24eb03\") " pod="openshift-dns/node-resolver-kd62p"
Jul 10 22:55:40.622825 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622100    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3734910e-9249-4060-afc4-55140bcb9911-dbus\") pod \"global-pull-secret-syncer-7zb67\" (UID: \"3734910e-9249-4060-afc4-55140bcb9911\") " pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:55:40.622825 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622112    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-run-systemd\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.622825 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622059    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-run-openvswitch\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.622825 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622204    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btjmf\" (UniqueName: \"kubernetes.io/projected/cdbc917d-2667-438e-928b-326933a924d8-kube-api-access-btjmf\") pod \"multus-additional-cni-plugins-xpkzr\" (UID: \"cdbc917d-2667-438e-928b-326933a924d8\") " pod="openshift-multus/multus-additional-cni-plugins-xpkzr"
Jul 10 22:55:40.622825 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622234    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0ee6c937-5bad-4104-916a-71224add52b5-konnectivity-ca\") pod \"konnectivity-agent-fwk45\" (UID: \"0ee6c937-5bad-4104-916a-71224add52b5\") " pod="kube-system/konnectivity-agent-fwk45"
Jul 10 22:55:40.622825 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622255    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-host-slash\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.622825 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622278    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-env-overrides\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.622825 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622301    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cdbc917d-2667-438e-928b-326933a924d8-system-cni-dir\") pod \"multus-additional-cni-plugins-xpkzr\" (UID: \"cdbc917d-2667-438e-928b-326933a924d8\") " pod="openshift-multus/multus-additional-cni-plugins-xpkzr"
Jul 10 22:55:40.623276 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622348    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cdbc917d-2667-438e-928b-326933a924d8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xpkzr\" (UID: \"cdbc917d-2667-438e-928b-326933a924d8\") " pod="openshift-multus/multus-additional-cni-plugins-xpkzr"
Jul 10 22:55:40.623276 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622393    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3734910e-9249-4060-afc4-55140bcb9911-kubelet-config\") pod \"global-pull-secret-syncer-7zb67\" (UID: \"3734910e-9249-4060-afc4-55140bcb9911\") " pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:55:40.623276 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622415    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krzx9\" (UniqueName: \"kubernetes.io/projected/257b0f09-db72-43e3-a3d5-a28a4fcdc3d6-kube-api-access-krzx9\") pod \"aws-ebs-csi-driver-node-xbkks\" (UID: \"257b0f09-db72-43e3-a3d5-a28a4fcdc3d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks"
Jul 10 22:55:40.623276 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622470    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-etc-kubernetes\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.623276 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622494    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-host\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.623276 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622517    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e17cacf8-0106-484f-a226-2f584a8fc5a5-host\") pod \"node-ca-298xj\" (UID: \"e17cacf8-0106-484f-a226-2f584a8fc5a5\") " pod="openshift-image-registry/node-ca-298xj"
Jul 10 22:55:40.623276 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622540    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-lib-modules\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.623276 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622565    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-log-socket\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.623276 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622578    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3734910e-9249-4060-afc4-55140bcb9911-kubelet-config\") pod \"global-pull-secret-syncer-7zb67\" (UID: \"3734910e-9249-4060-afc4-55140bcb9911\") " pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:55:40.623276 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622588    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-host-run-ovn-kubernetes\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.623276 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622611    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-host-slash\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.623276 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622613    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfbs6\" (UniqueName: \"kubernetes.io/projected/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-kube-api-access-jfbs6\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.623276 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622640    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-multus-socket-dir-parent\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.623276 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622665    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4szdb\" (UniqueName: \"kubernetes.io/projected/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-kube-api-access-4szdb\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.623276 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622713    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/50a4a720-01af-4105-9de4-260ee26283a4-etc-tuned\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.623276 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622741    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54d84e33-5ba4-4b6c-8c8f-4ddaac24eb03-tmp-dir\") pod \"node-resolver-kd62p\" (UID: \"54d84e33-5ba4-4b6c-8c8f-4ddaac24eb03\") " pod="openshift-dns/node-resolver-kd62p"
Jul 10 22:55:40.623276 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622769    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cdbc917d-2667-438e-928b-326933a924d8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xpkzr\" (UID: \"cdbc917d-2667-438e-928b-326933a924d8\") " pod="openshift-multus/multus-additional-cni-plugins-xpkzr"
Jul 10 22:55:40.623900 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622794    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cdbc917d-2667-438e-928b-326933a924d8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xpkzr\" (UID: \"cdbc917d-2667-438e-928b-326933a924d8\") " pod="openshift-multus/multus-additional-cni-plugins-xpkzr"
Jul 10 22:55:40.623900 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622846    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-etc-sysconfig\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.623900 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622847    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0ee6c937-5bad-4104-916a-71224add52b5-konnectivity-ca\") pod \"konnectivity-agent-fwk45\" (UID: \"0ee6c937-5bad-4104-916a-71224add52b5\") " pod="kube-system/konnectivity-agent-fwk45"
Jul 10 22:55:40.623900 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622873    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-etc-systemd\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.623900 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622898    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-ovn-node-metrics-cert\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.623900 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622921    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e17cacf8-0106-484f-a226-2f584a8fc5a5-serviceca\") pod \"node-ca-298xj\" (UID: \"e17cacf8-0106-484f-a226-2f584a8fc5a5\") " pod="openshift-image-registry/node-ca-298xj"
Jul 10 22:55:40.623900 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622941    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-env-overrides\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.623900 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622944    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xmms\" (UniqueName: \"kubernetes.io/projected/e17cacf8-0106-484f-a226-2f584a8fc5a5-kube-api-access-7xmms\") pod \"node-ca-298xj\" (UID: \"e17cacf8-0106-484f-a226-2f584a8fc5a5\") " pod="openshift-image-registry/node-ca-298xj"
Jul 10 22:55:40.623900 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622976    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-run\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.623900 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.622995    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-system-cni-dir\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.623900 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623010    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-host-run-netns\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.623900 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623024    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-host-var-lib-cni-bin\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.623900 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623038    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-multus-conf-dir\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.623900 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623053    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0ee6c937-5bad-4104-916a-71224add52b5-agent-certs\") pod \"konnectivity-agent-fwk45\" (UID: \"0ee6c937-5bad-4104-916a-71224add52b5\") " pod="kube-system/konnectivity-agent-fwk45"
Jul 10 22:55:40.623900 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623067    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-etc-sysctl-d\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.623900 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623084    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-host-cni-netd\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.623900 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623091    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-multus-socket-dir-parent\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.623900 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623098    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-ovnkube-config\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.624737 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623117    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cdbc917d-2667-438e-928b-326933a924d8-cnibin\") pod \"multus-additional-cni-plugins-xpkzr\" (UID: \"cdbc917d-2667-438e-928b-326933a924d8\") " pod="openshift-multus/multus-additional-cni-plugins-xpkzr"
Jul 10 22:55:40.624737 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623131    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-hostroot\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.624737 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623148    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-etc-openvswitch\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.624737 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623180    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/257b0f09-db72-43e3-a3d5-a28a4fcdc3d6-socket-dir\") pod \"aws-ebs-csi-driver-node-xbkks\" (UID: \"257b0f09-db72-43e3-a3d5-a28a4fcdc3d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks"
Jul 10 22:55:40.624737 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623197    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/257b0f09-db72-43e3-a3d5-a28a4fcdc3d6-device-dir\") pod \"aws-ebs-csi-driver-node-xbkks\" (UID: \"257b0f09-db72-43e3-a3d5-a28a4fcdc3d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks"
Jul 10 22:55:40.624737 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623211    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjqs6\" (UniqueName: \"kubernetes.io/projected/9829a27f-3dbd-44c7-b278-f26beb69f82a-kube-api-access-gjqs6\") pod \"network-metrics-daemon-w9kxw\" (UID: \"9829a27f-3dbd-44c7-b278-f26beb69f82a\") " pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:55:40.624737 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623227    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-host-var-lib-cni-multus\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.624737 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623244    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/257b0f09-db72-43e3-a3d5-a28a4fcdc3d6-registration-dir\") pod \"aws-ebs-csi-driver-node-xbkks\" (UID: \"257b0f09-db72-43e3-a3d5-a28a4fcdc3d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks"
Jul 10 22:55:40.624737 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623262    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-var-lib-openvswitch\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.624737 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623269    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-etc-kubernetes\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.624737 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623279    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-run-ovn\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.624737 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623306    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-run-ovn\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.624737 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623310    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-node-log\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.624737 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623344    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-host-cni-bin\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.624737 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623423    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cdbc917d-2667-438e-928b-326933a924d8-cni-binary-copy\") pod \"multus-additional-cni-plugins-xpkzr\" (UID: \"cdbc917d-2667-438e-928b-326933a924d8\") " pod="openshift-multus/multus-additional-cni-plugins-xpkzr"
Jul 10 22:55:40.624737 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623461    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-host-run-multus-certs\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.624737 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623489    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/257b0f09-db72-43e3-a3d5-a28a4fcdc3d6-etc-selinux\") pod \"aws-ebs-csi-driver-node-xbkks\" (UID: \"257b0f09-db72-43e3-a3d5-a28a4fcdc3d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks"
Jul 10 22:55:40.625608 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623517    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-etc-modprobe-d\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.625608 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623549    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5x9d\" (UniqueName: \"kubernetes.io/projected/54d84e33-5ba4-4b6c-8c8f-4ddaac24eb03-kube-api-access-t5x9d\") pod \"node-resolver-kd62p\" (UID: \"54d84e33-5ba4-4b6c-8c8f-4ddaac24eb03\") " pod="openshift-dns/node-resolver-kd62p"
Jul 10 22:55:40.625608 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623573    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-ovnkube-script-lib\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.625608 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623598    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3fe0a8af-e7c7-47cf-81d5-5e0c3c0ad4e2-iptables-alerter-script\") pod \"iptables-alerter-hvkfr\" (UID: \"3fe0a8af-e7c7-47cf-81d5-5e0c3c0ad4e2\") " pod="openshift-network-operator/iptables-alerter-hvkfr"
Jul 10 22:55:40.625608 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623626    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-cni-binary-copy\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.625608 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623615    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 10 22:55:40.625608 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623653    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-etc-sysctl-conf\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.625608 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623676    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/50a4a720-01af-4105-9de4-260ee26283a4-tmp\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.625608 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623699    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-etc-kubernetes\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.625608 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623722    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-host-run-netns\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.625608 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623741    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-cnibin\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.625608 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623762    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-host-run-k8s-cni-cncf-io\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.625608 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623838    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-multus-daemon-config\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.625608 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623856    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54d84e33-5ba4-4b6c-8c8f-4ddaac24eb03-tmp-dir\") pod \"node-resolver-kd62p\" (UID: \"54d84e33-5ba4-4b6c-8c8f-4ddaac24eb03\") " pod="openshift-dns/node-resolver-kd62p"
Jul 10 22:55:40.625608 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623922    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cdbc917d-2667-438e-928b-326933a924d8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xpkzr\" (UID: \"cdbc917d-2667-438e-928b-326933a924d8\") " pod="openshift-multus/multus-additional-cni-plugins-xpkzr"
Jul 10 22:55:40.625608 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623978    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-etc-systemd\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.625608 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.624014    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-node-log\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.625608 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.624059    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-host-cni-bin\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.626664 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.623347    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cdbc917d-2667-438e-928b-326933a924d8-system-cni-dir\") pod \"multus-additional-cni-plugins-xpkzr\" (UID: \"cdbc917d-2667-438e-928b-326933a924d8\") " pod="openshift-multus/multus-additional-cni-plugins-xpkzr"
Jul 10 22:55:40.626664 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.624223    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-cnibin\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.626664 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.624389    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e17cacf8-0106-484f-a226-2f584a8fc5a5-serviceca\") pod \"node-ca-298xj\" (UID: \"e17cacf8-0106-484f-a226-2f584a8fc5a5\") " pod="openshift-image-registry/node-ca-298xj"
Jul 10 22:55:40.626664 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.624433    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-log-socket\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.626664 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.624515    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-etc-modprobe-d\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.626664 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.624553    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e17cacf8-0106-484f-a226-2f584a8fc5a5-host\") pod \"node-ca-298xj\" (UID: \"e17cacf8-0106-484f-a226-2f584a8fc5a5\") " pod="openshift-image-registry/node-ca-298xj"
Jul 10 22:55:40.626664 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.624555    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cdbc917d-2667-438e-928b-326933a924d8-cni-binary-copy\") pod \"multus-additional-cni-plugins-xpkzr\" (UID: \"cdbc917d-2667-438e-928b-326933a924d8\") " pod="openshift-multus/multus-additional-cni-plugins-xpkzr"
Jul 10 22:55:40.626664 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.624581    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-ovnkube-script-lib\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.626664 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.624631    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-host-run-k8s-cni-cncf-io\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.626664 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.624645    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/257b0f09-db72-43e3-a3d5-a28a4fcdc3d6-etc-selinux\") pod \"aws-ebs-csi-driver-node-xbkks\" (UID: \"257b0f09-db72-43e3-a3d5-a28a4fcdc3d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks"
Jul 10 22:55:40.626664 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.624769    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-etc-sysctl-conf\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.626664 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.624808    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3fe0a8af-e7c7-47cf-81d5-5e0c3c0ad4e2-iptables-alerter-script\") pod \"iptables-alerter-hvkfr\" (UID: \"3fe0a8af-e7c7-47cf-81d5-5e0c3c0ad4e2\") " pod="openshift-network-operator/iptables-alerter-hvkfr"
Jul 10 22:55:40.626664 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.624832    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-etc-kubernetes\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.626664 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.624863    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-lib-modules\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.626664 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.624877    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-host-cni-netd\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.626664 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.624952    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/257b0f09-db72-43e3-a3d5-a28a4fcdc3d6-registration-dir\") pod \"aws-ebs-csi-driver-node-xbkks\" (UID: \"257b0f09-db72-43e3-a3d5-a28a4fcdc3d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks"
Jul 10 22:55:40.626664 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.624995    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-host-var-lib-cni-multus\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.626664 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.625033    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-hostroot\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.627564 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.625046    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-var-lib-openvswitch\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.627564 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.625072    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-multus-conf-dir\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.627564 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.625091    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-host-run-ovn-kubernetes\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.627564 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.625141    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-run\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.627564 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.625183    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-system-cni-dir\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.627564 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.625213    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cdbc917d-2667-438e-928b-326933a924d8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xpkzr\" (UID: \"cdbc917d-2667-438e-928b-326933a924d8\") " pod="openshift-multus/multus-additional-cni-plugins-xpkzr"
Jul 10 22:55:40.627564 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.625215    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-host-run-netns\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.627564 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.625224    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-multus-daemon-config\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.627564 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.625242    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-host-var-lib-cni-bin\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.627564 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.625277    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-host-run-netns\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.627564 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.625316    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-host\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.627564 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.625375    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-etc-sysconfig\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.627564 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.625356    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cdbc917d-2667-438e-928b-326933a924d8-cnibin\") pod \"multus-additional-cni-plugins-xpkzr\" (UID: \"cdbc917d-2667-438e-928b-326933a924d8\") " pod="openshift-multus/multus-additional-cni-plugins-xpkzr"
Jul 10 22:55:40.627564 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.625387    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-ovnkube-config\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.627564 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.625413    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-etc-openvswitch\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.627564 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.625423    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-host-run-multus-certs\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.627564 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.625482    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/257b0f09-db72-43e3-a3d5-a28a4fcdc3d6-device-dir\") pod \"aws-ebs-csi-driver-node-xbkks\" (UID: \"257b0f09-db72-43e3-a3d5-a28a4fcdc3d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks"
Jul 10 22:55:40.627564 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.625534    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/257b0f09-db72-43e3-a3d5-a28a4fcdc3d6-socket-dir\") pod \"aws-ebs-csi-driver-node-xbkks\" (UID: \"257b0f09-db72-43e3-a3d5-a28a4fcdc3d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks"
Jul 10 22:55:40.628167 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.625631    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/50a4a720-01af-4105-9de4-260ee26283a4-etc-sysctl-d\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.628167 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.626215    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cdbc917d-2667-438e-928b-326933a924d8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xpkzr\" (UID: \"cdbc917d-2667-438e-928b-326933a924d8\") " pod="openshift-multus/multus-additional-cni-plugins-xpkzr"
Jul 10 22:55:40.628167 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.626215    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-cni-binary-copy\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.628167 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.627336    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/50a4a720-01af-4105-9de4-260ee26283a4-etc-tuned\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.628167 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.627544    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-ovn-node-metrics-cert\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.628167 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.627984    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/50a4a720-01af-4105-9de4-260ee26283a4-tmp\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.628167 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.628035    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0ee6c937-5bad-4104-916a-71224add52b5-agent-certs\") pod \"konnectivity-agent-fwk45\" (UID: \"0ee6c937-5bad-4104-916a-71224add52b5\") " pod="kube-system/konnectivity-agent-fwk45"
Jul 10 22:55:40.631142 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:40.631122    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 22:55:40.631191 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:40.631154    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 22:55:40.631191 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:40.631168    2563 projected.go:196] Error preparing data for projected volume kube-api-access-j8924 for pod openshift-network-diagnostics/network-check-target-8nt9g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 22:55:40.631254 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:40.631238    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3f0e1c5a-a319-437d-9aef-8b3c4150a3b1-kube-api-access-j8924 podName:3f0e1c5a-a319-437d-9aef-8b3c4150a3b1 nodeName:}" failed. No retries permitted until 2026-07-10 22:55:41.131219164 +0000 UTC m=+2.017955307 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-j8924" (UniqueName: "kubernetes.io/projected/3f0e1c5a-a319-437d-9aef-8b3c4150a3b1-kube-api-access-j8924") pod "network-check-target-8nt9g" (UID: "3f0e1c5a-a319-437d-9aef-8b3c4150a3b1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 22:55:40.634168 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.634133    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrzl5\" (UniqueName: \"kubernetes.io/projected/3fe0a8af-e7c7-47cf-81d5-5e0c3c0ad4e2-kube-api-access-xrzl5\") pod \"iptables-alerter-hvkfr\" (UID: \"3fe0a8af-e7c7-47cf-81d5-5e0c3c0ad4e2\") " pod="openshift-network-operator/iptables-alerter-hvkfr"
Jul 10 22:55:40.634665 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.634552    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btjmf\" (UniqueName: \"kubernetes.io/projected/cdbc917d-2667-438e-928b-326933a924d8-kube-api-access-btjmf\") pod \"multus-additional-cni-plugins-xpkzr\" (UID: \"cdbc917d-2667-438e-928b-326933a924d8\") " pod="openshift-multus/multus-additional-cni-plugins-xpkzr"
Jul 10 22:55:40.634783 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.634742    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdnkc\" (UniqueName: \"kubernetes.io/projected/50a4a720-01af-4105-9de4-260ee26283a4-kube-api-access-zdnkc\") pod \"tuned-w2wjh\" (UID: \"50a4a720-01af-4105-9de4-260ee26283a4\") " pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.636144 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.636120    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5x9d\" (UniqueName: \"kubernetes.io/projected/54d84e33-5ba4-4b6c-8c8f-4ddaac24eb03-kube-api-access-t5x9d\") pod \"node-resolver-kd62p\" (UID: \"54d84e33-5ba4-4b6c-8c8f-4ddaac24eb03\") " pod="openshift-dns/node-resolver-kd62p"
Jul 10 22:55:40.636253 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.636189    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krzx9\" (UniqueName: \"kubernetes.io/projected/257b0f09-db72-43e3-a3d5-a28a4fcdc3d6-kube-api-access-krzx9\") pod \"aws-ebs-csi-driver-node-xbkks\" (UID: \"257b0f09-db72-43e3-a3d5-a28a4fcdc3d6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks"
Jul 10 22:55:40.636377 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.636339    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xmms\" (UniqueName: \"kubernetes.io/projected/e17cacf8-0106-484f-a226-2f584a8fc5a5-kube-api-access-7xmms\") pod \"node-ca-298xj\" (UID: \"e17cacf8-0106-484f-a226-2f584a8fc5a5\") " pod="openshift-image-registry/node-ca-298xj"
Jul 10 22:55:40.636572 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.636550    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4szdb\" (UniqueName: \"kubernetes.io/projected/b8382cd0-6bcd-4037-814a-dfa9c652d3bd-kube-api-access-4szdb\") pod \"multus-rdxmp\" (UID: \"b8382cd0-6bcd-4037-814a-dfa9c652d3bd\") " pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.636812 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.636797    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfbs6\" (UniqueName: \"kubernetes.io/projected/861ff3e2-b6a3-4a8b-a23f-88faf0b7e203-kube-api-access-jfbs6\") pod \"ovnkube-node-qj86t\" (UID: \"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203\") " pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.637203 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.637186    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjqs6\" (UniqueName: \"kubernetes.io/projected/9829a27f-3dbd-44c7-b278-f26beb69f82a-kube-api-access-gjqs6\") pod \"network-metrics-daemon-w9kxw\" (UID: \"9829a27f-3dbd-44c7-b278-f26beb69f82a\") " pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:55:40.681218 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:40.681181    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98bd729d123769a48c9d58a5f9915bea.slice/crio-c85deb35b0399a3f79121b17ce29dadce344402d70e39d49d9d9ab6ebe186bdf WatchSource:0}: Error finding container c85deb35b0399a3f79121b17ce29dadce344402d70e39d49d9d9ab6ebe186bdf: Status 404 returned error can't find the container with id c85deb35b0399a3f79121b17ce29dadce344402d70e39d49d9d9ab6ebe186bdf
Jul 10 22:55:40.686224 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:40.686200    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c99c1f4b605d867ddb0d2d31d3a10b5.slice/crio-c566727b882004ddc61715f7b678382b3c162a5761965bab7fcd37905aa6ac4d WatchSource:0}: Error finding container c566727b882004ddc61715f7b678382b3c162a5761965bab7fcd37905aa6ac4d: Status 404 returned error can't find the container with id c566727b882004ddc61715f7b678382b3c162a5761965bab7fcd37905aa6ac4d
Jul 10 22:55:40.687240 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.687219    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 22:55:40.748285 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.748190    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-298xj"
Jul 10 22:55:40.757924 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:40.757894    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode17cacf8_0106_484f_a226_2f584a8fc5a5.slice/crio-44e4d16fc0d99c5f6f88fc52775c0aec00d15b3931199b953b5aaea90e2a4c9a WatchSource:0}: Error finding container 44e4d16fc0d99c5f6f88fc52775c0aec00d15b3931199b953b5aaea90e2a4c9a: Status 404 returned error can't find the container with id 44e4d16fc0d99c5f6f88fc52775c0aec00d15b3931199b953b5aaea90e2a4c9a
Jul 10 22:55:40.766665 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.766636    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-w2wjh"
Jul 10 22:55:40.775090 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:40.775064    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50a4a720_01af_4105_9de4_260ee26283a4.slice/crio-e0a79145c05e988b3cf6cfa804cb3023893fed6e99671acde196fdcc3ab8f1a5 WatchSource:0}: Error finding container e0a79145c05e988b3cf6cfa804cb3023893fed6e99671acde196fdcc3ab8f1a5: Status 404 returned error can't find the container with id e0a79145c05e988b3cf6cfa804cb3023893fed6e99671acde196fdcc3ab8f1a5
Jul 10 22:55:40.785767 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.785732    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kd62p"
Jul 10 22:55:40.795073 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:40.795041    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54d84e33_5ba4_4b6c_8c8f_4ddaac24eb03.slice/crio-42c02594a07ddb64701c2f5b04c33343d6a30ce02263085417ddd9afc2807307 WatchSource:0}: Error finding container 42c02594a07ddb64701c2f5b04c33343d6a30ce02263085417ddd9afc2807307: Status 404 returned error can't find the container with id 42c02594a07ddb64701c2f5b04c33343d6a30ce02263085417ddd9afc2807307
Jul 10 22:55:40.804457 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.804431    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:55:40.812546 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.812520    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xpkzr"
Jul 10 22:55:40.813397 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:40.813353    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod861ff3e2_b6a3_4a8b_a23f_88faf0b7e203.slice/crio-14667d89e5265e83bd9061deffedc068820a1797eec9af98a62712220da45498 WatchSource:0}: Error finding container 14667d89e5265e83bd9061deffedc068820a1797eec9af98a62712220da45498: Status 404 returned error can't find the container with id 14667d89e5265e83bd9061deffedc068820a1797eec9af98a62712220da45498
Jul 10 22:55:40.822096 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:40.822068    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdbc917d_2667_438e_928b_326933a924d8.slice/crio-681081cee3d72f2e5a914fbd98463732cb5fbe1f9bdafdf9dd34c6a5dec5a934 WatchSource:0}: Error finding container 681081cee3d72f2e5a914fbd98463732cb5fbe1f9bdafdf9dd34c6a5dec5a934: Status 404 returned error can't find the container with id 681081cee3d72f2e5a914fbd98463732cb5fbe1f9bdafdf9dd34c6a5dec5a934
Jul 10 22:55:40.842290 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.842256    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hvkfr"
Jul 10 22:55:40.850893 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:40.850860    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fe0a8af_e7c7_47cf_81d5_5e0c3c0ad4e2.slice/crio-cd8135463658f9a994e0bc0e1edde5002f2581fa461bd3493ea07ec30b6dfe07 WatchSource:0}: Error finding container cd8135463658f9a994e0bc0e1edde5002f2581fa461bd3493ea07ec30b6dfe07: Status 404 returned error can't find the container with id cd8135463658f9a994e0bc0e1edde5002f2581fa461bd3493ea07ec30b6dfe07
Jul 10 22:55:40.859401 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.859373    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fwk45"
Jul 10 22:55:40.868664 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:40.868635    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ee6c937_5bad_4104_916a_71224add52b5.slice/crio-ccd84d5341a161a93a209cba46af5e39cba224bf690ad0fccc6a782ad7a14800 WatchSource:0}: Error finding container ccd84d5341a161a93a209cba46af5e39cba224bf690ad0fccc6a782ad7a14800: Status 404 returned error can't find the container with id ccd84d5341a161a93a209cba46af5e39cba224bf690ad0fccc6a782ad7a14800
Jul 10 22:55:40.870476 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.870450    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 22:55:40.870676 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.870654    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 22:55:40.880127 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.880100    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks"
Jul 10 22:55:40.884996 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:40.884978    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rdxmp"
Jul 10 22:55:40.891265 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:40.891235    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod257b0f09_db72_43e3_a3d5_a28a4fcdc3d6.slice/crio-635c5e699d3fa71c498f1266d9c946363f99a5c3d436c4cea8d1ab171135da92 WatchSource:0}: Error finding container 635c5e699d3fa71c498f1266d9c946363f99a5c3d436c4cea8d1ab171135da92: Status 404 returned error can't find the container with id 635c5e699d3fa71c498f1266d9c946363f99a5c3d436c4cea8d1ab171135da92
Jul 10 22:55:40.893991 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:55:40.893965    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8382cd0_6bcd_4037_814a_dfa9c652d3bd.slice/crio-babebff21119efb366bdea3988d1f1b77ef7a1b6d98274a9354582d6ad7159e6 WatchSource:0}: Error finding container babebff21119efb366bdea3988d1f1b77ef7a1b6d98274a9354582d6ad7159e6: Status 404 returned error can't find the container with id babebff21119efb366bdea3988d1f1b77ef7a1b6d98274a9354582d6ad7159e6
Jul 10 22:55:41.128346 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:41.128255    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3734910e-9249-4060-afc4-55140bcb9911-original-pull-secret\") pod \"global-pull-secret-syncer-7zb67\" (UID: \"3734910e-9249-4060-afc4-55140bcb9911\") " pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:55:41.128346 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:41.128299    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs\") pod \"network-metrics-daemon-w9kxw\" (UID: \"9829a27f-3dbd-44c7-b278-f26beb69f82a\") " pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:55:41.128591 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:41.128456    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 22:55:41.128591 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:41.128457    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 22:55:41.128591 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:41.128523    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs podName:9829a27f-3dbd-44c7-b278-f26beb69f82a nodeName:}" failed. No retries permitted until 2026-07-10 22:55:42.128506449 +0000 UTC m=+3.015242572 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs") pod "network-metrics-daemon-w9kxw" (UID: "9829a27f-3dbd-44c7-b278-f26beb69f82a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 22:55:41.128591 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:41.128538    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3734910e-9249-4060-afc4-55140bcb9911-original-pull-secret podName:3734910e-9249-4060-afc4-55140bcb9911 nodeName:}" failed. No retries permitted until 2026-07-10 22:55:42.128532439 +0000 UTC m=+3.015268559 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3734910e-9249-4060-afc4-55140bcb9911-original-pull-secret") pod "global-pull-secret-syncer-7zb67" (UID: "3734910e-9249-4060-afc4-55140bcb9911") : object "kube-system"/"original-pull-secret" not registered
Jul 10 22:55:41.229673 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:41.229630    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8924\" (UniqueName: \"kubernetes.io/projected/3f0e1c5a-a319-437d-9aef-8b3c4150a3b1-kube-api-access-j8924\") pod \"network-check-target-8nt9g\" (UID: \"3f0e1c5a-a319-437d-9aef-8b3c4150a3b1\") " pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:55:41.229976 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:41.229910    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 22:55:41.229976 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:41.229935    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 22:55:41.229976 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:41.229949    2563 projected.go:196] Error preparing data for projected volume kube-api-access-j8924 for pod openshift-network-diagnostics/network-check-target-8nt9g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 22:55:41.230176 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:41.230015    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3f0e1c5a-a319-437d-9aef-8b3c4150a3b1-kube-api-access-j8924 podName:3f0e1c5a-a319-437d-9aef-8b3c4150a3b1 nodeName:}" failed. No retries permitted until 2026-07-10 22:55:42.229997023 +0000 UTC m=+3.116733149 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-j8924" (UniqueName: "kubernetes.io/projected/3f0e1c5a-a319-437d-9aef-8b3c4150a3b1-kube-api-access-j8924") pod "network-check-target-8nt9g" (UID: "3f0e1c5a-a319-437d-9aef-8b3c4150a3b1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 22:55:41.462484 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:41.462274    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 22:50:40 +0000 UTC" deadline="2028-03-07 22:38:09.018303509 +0000 UTC"
Jul 10 22:55:41.462484 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:41.462338    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14543h42m27.555969607s"
Jul 10 22:55:41.553078 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:41.552962    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:55:41.553249 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:41.553133    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nt9g" podUID="3f0e1c5a-a319-437d-9aef-8b3c4150a3b1"
Jul 10 22:55:41.583563 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:41.583495    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xpkzr" event={"ID":"cdbc917d-2667-438e-928b-326933a924d8","Type":"ContainerStarted","Data":"681081cee3d72f2e5a914fbd98463732cb5fbe1f9bdafdf9dd34c6a5dec5a934"}
Jul 10 22:55:41.610802 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:41.610760    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-239.ec2.internal" event={"ID":"6c99c1f4b605d867ddb0d2d31d3a10b5","Type":"ContainerStarted","Data":"c566727b882004ddc61715f7b678382b3c162a5761965bab7fcd37905aa6ac4d"}
Jul 10 22:55:41.640702 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:41.640589    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rdxmp" event={"ID":"b8382cd0-6bcd-4037-814a-dfa9c652d3bd","Type":"ContainerStarted","Data":"babebff21119efb366bdea3988d1f1b77ef7a1b6d98274a9354582d6ad7159e6"}
Jul 10 22:55:41.651612 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:41.651568    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks" event={"ID":"257b0f09-db72-43e3-a3d5-a28a4fcdc3d6","Type":"ContainerStarted","Data":"635c5e699d3fa71c498f1266d9c946363f99a5c3d436c4cea8d1ab171135da92"}
Jul 10 22:55:41.660395 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:41.660032    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qj86t" event={"ID":"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203","Type":"ContainerStarted","Data":"14667d89e5265e83bd9061deffedc068820a1797eec9af98a62712220da45498"}
Jul 10 22:55:41.682155 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:41.682015    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kd62p" event={"ID":"54d84e33-5ba4-4b6c-8c8f-4ddaac24eb03","Type":"ContainerStarted","Data":"42c02594a07ddb64701c2f5b04c33343d6a30ce02263085417ddd9afc2807307"}
Jul 10 22:55:41.696456 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:41.696405    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-w2wjh" event={"ID":"50a4a720-01af-4105-9de4-260ee26283a4","Type":"ContainerStarted","Data":"e0a79145c05e988b3cf6cfa804cb3023893fed6e99671acde196fdcc3ab8f1a5"}
Jul 10 22:55:41.701160 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:41.701072    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-298xj" event={"ID":"e17cacf8-0106-484f-a226-2f584a8fc5a5","Type":"ContainerStarted","Data":"44e4d16fc0d99c5f6f88fc52775c0aec00d15b3931199b953b5aaea90e2a4c9a"}
Jul 10 22:55:41.716981 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:41.716885    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-239.ec2.internal" event={"ID":"98bd729d123769a48c9d58a5f9915bea","Type":"ContainerStarted","Data":"c85deb35b0399a3f79121b17ce29dadce344402d70e39d49d9d9ab6ebe186bdf"}
Jul 10 22:55:41.729118 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:41.729074    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fwk45" event={"ID":"0ee6c937-5bad-4104-916a-71224add52b5","Type":"ContainerStarted","Data":"ccd84d5341a161a93a209cba46af5e39cba224bf690ad0fccc6a782ad7a14800"}
Jul 10 22:55:41.735725 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:41.735680    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hvkfr" event={"ID":"3fe0a8af-e7c7-47cf-81d5-5e0c3c0ad4e2","Type":"ContainerStarted","Data":"cd8135463658f9a994e0bc0e1edde5002f2581fa461bd3493ea07ec30b6dfe07"}
Jul 10 22:55:41.870250 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:41.870216    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 22:55:42.136309 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:42.136214    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3734910e-9249-4060-afc4-55140bcb9911-original-pull-secret\") pod \"global-pull-secret-syncer-7zb67\" (UID: \"3734910e-9249-4060-afc4-55140bcb9911\") " pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:55:42.136309 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:42.136275    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs\") pod \"network-metrics-daemon-w9kxw\" (UID: \"9829a27f-3dbd-44c7-b278-f26beb69f82a\") " pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:55:42.136568 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:42.136449    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 22:55:42.136568 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:42.136465    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 22:55:42.136568 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:42.136521    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs podName:9829a27f-3dbd-44c7-b278-f26beb69f82a nodeName:}" failed. No retries permitted until 2026-07-10 22:55:44.13650281 +0000 UTC m=+5.023238949 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs") pod "network-metrics-daemon-w9kxw" (UID: "9829a27f-3dbd-44c7-b278-f26beb69f82a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 22:55:42.136568 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:42.136539    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3734910e-9249-4060-afc4-55140bcb9911-original-pull-secret podName:3734910e-9249-4060-afc4-55140bcb9911 nodeName:}" failed. No retries permitted until 2026-07-10 22:55:44.136530492 +0000 UTC m=+5.023266615 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3734910e-9249-4060-afc4-55140bcb9911-original-pull-secret") pod "global-pull-secret-syncer-7zb67" (UID: "3734910e-9249-4060-afc4-55140bcb9911") : object "kube-system"/"original-pull-secret" not registered
Jul 10 22:55:42.237239 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:42.237198    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8924\" (UniqueName: \"kubernetes.io/projected/3f0e1c5a-a319-437d-9aef-8b3c4150a3b1-kube-api-access-j8924\") pod \"network-check-target-8nt9g\" (UID: \"3f0e1c5a-a319-437d-9aef-8b3c4150a3b1\") " pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:55:42.237700 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:42.237435    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 22:55:42.237700 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:42.237456    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 22:55:42.237700 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:42.237470    2563 projected.go:196] Error preparing data for projected volume kube-api-access-j8924 for pod openshift-network-diagnostics/network-check-target-8nt9g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 22:55:42.237700 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:42.237535    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3f0e1c5a-a319-437d-9aef-8b3c4150a3b1-kube-api-access-j8924 podName:3f0e1c5a-a319-437d-9aef-8b3c4150a3b1 nodeName:}" failed. No retries permitted until 2026-07-10 22:55:44.237517043 +0000 UTC m=+5.124253180 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-j8924" (UniqueName: "kubernetes.io/projected/3f0e1c5a-a319-437d-9aef-8b3c4150a3b1-kube-api-access-j8924") pod "network-check-target-8nt9g" (UID: "3f0e1c5a-a319-437d-9aef-8b3c4150a3b1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 22:55:42.463139 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:42.463041    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 22:50:40 +0000 UTC" deadline="2027-12-13 14:10:24.785951026 +0000 UTC"
Jul 10 22:55:42.463139 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:42.463087    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12495h14m42.322868652s"
Jul 10 22:55:42.550629 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:42.550566    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:55:42.550831 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:42.550811    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zb67" podUID="3734910e-9249-4060-afc4-55140bcb9911"
Jul 10 22:55:42.551229 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:42.551210    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:55:42.551338 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:42.551314    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9kxw" podUID="9829a27f-3dbd-44c7-b278-f26beb69f82a"
Jul 10 22:55:42.971498 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:42.971466    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 22:55:43.553313 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:43.553282    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:55:43.553881 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:43.553428    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nt9g" podUID="3f0e1c5a-a319-437d-9aef-8b3c4150a3b1"
Jul 10 22:55:44.155564 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:44.154663    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3734910e-9249-4060-afc4-55140bcb9911-original-pull-secret\") pod \"global-pull-secret-syncer-7zb67\" (UID: \"3734910e-9249-4060-afc4-55140bcb9911\") " pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:55:44.155564 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:44.154723    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs\") pod \"network-metrics-daemon-w9kxw\" (UID: \"9829a27f-3dbd-44c7-b278-f26beb69f82a\") " pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:55:44.155564 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:44.154891    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 22:55:44.155564 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:44.154958    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs podName:9829a27f-3dbd-44c7-b278-f26beb69f82a nodeName:}" failed. No retries permitted until 2026-07-10 22:55:48.154938437 +0000 UTC m=+9.041674567 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs") pod "network-metrics-daemon-w9kxw" (UID: "9829a27f-3dbd-44c7-b278-f26beb69f82a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 22:55:44.155564 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:44.155465    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 22:55:44.155564 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:44.155521    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3734910e-9249-4060-afc4-55140bcb9911-original-pull-secret podName:3734910e-9249-4060-afc4-55140bcb9911 nodeName:}" failed. No retries permitted until 2026-07-10 22:55:48.155504237 +0000 UTC m=+9.042240363 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3734910e-9249-4060-afc4-55140bcb9911-original-pull-secret") pod "global-pull-secret-syncer-7zb67" (UID: "3734910e-9249-4060-afc4-55140bcb9911") : object "kube-system"/"original-pull-secret" not registered
Jul 10 22:55:44.255316 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:44.255271    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8924\" (UniqueName: \"kubernetes.io/projected/3f0e1c5a-a319-437d-9aef-8b3c4150a3b1-kube-api-access-j8924\") pod \"network-check-target-8nt9g\" (UID: \"3f0e1c5a-a319-437d-9aef-8b3c4150a3b1\") " pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:55:44.255524 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:44.255497    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 22:55:44.255524 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:44.255525    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 22:55:44.255652 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:44.255540    2563 projected.go:196] Error preparing data for projected volume kube-api-access-j8924 for pod openshift-network-diagnostics/network-check-target-8nt9g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 22:55:44.255652 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:44.255617    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3f0e1c5a-a319-437d-9aef-8b3c4150a3b1-kube-api-access-j8924 podName:3f0e1c5a-a319-437d-9aef-8b3c4150a3b1 nodeName:}" failed. No retries permitted until 2026-07-10 22:55:48.25559505 +0000 UTC m=+9.142331198 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-j8924" (UniqueName: "kubernetes.io/projected/3f0e1c5a-a319-437d-9aef-8b3c4150a3b1-kube-api-access-j8924") pod "network-check-target-8nt9g" (UID: "3f0e1c5a-a319-437d-9aef-8b3c4150a3b1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 22:55:44.550734 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:44.550639    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:55:44.550887 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:44.550817    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zb67" podUID="3734910e-9249-4060-afc4-55140bcb9911"
Jul 10 22:55:44.550887 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:44.550647    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:55:44.550984 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:44.550955    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9kxw" podUID="9829a27f-3dbd-44c7-b278-f26beb69f82a"
Jul 10 22:55:45.552228 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:45.552188    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:55:45.552748 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:45.552336    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nt9g" podUID="3f0e1c5a-a319-437d-9aef-8b3c4150a3b1"
Jul 10 22:55:46.550605 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:46.549895    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:55:46.550605 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:46.550028    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zb67" podUID="3734910e-9249-4060-afc4-55140bcb9911"
Jul 10 22:55:46.550605 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:46.550449    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:55:46.550605 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:46.550556    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9kxw" podUID="9829a27f-3dbd-44c7-b278-f26beb69f82a"
Jul 10 22:55:47.551802 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:47.551556    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:55:47.552232 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:47.551918    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nt9g" podUID="3f0e1c5a-a319-437d-9aef-8b3c4150a3b1"
Jul 10 22:55:48.186299 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:48.186258    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3734910e-9249-4060-afc4-55140bcb9911-original-pull-secret\") pod \"global-pull-secret-syncer-7zb67\" (UID: \"3734910e-9249-4060-afc4-55140bcb9911\") " pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:55:48.186511 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:48.186316    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs\") pod \"network-metrics-daemon-w9kxw\" (UID: \"9829a27f-3dbd-44c7-b278-f26beb69f82a\") " pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:55:48.186511 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:48.186421    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 22:55:48.186511 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:48.186448    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 22:55:48.186511 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:48.186502    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3734910e-9249-4060-afc4-55140bcb9911-original-pull-secret podName:3734910e-9249-4060-afc4-55140bcb9911 nodeName:}" failed. No retries permitted until 2026-07-10 22:55:56.186480799 +0000 UTC m=+17.073216928 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3734910e-9249-4060-afc4-55140bcb9911-original-pull-secret") pod "global-pull-secret-syncer-7zb67" (UID: "3734910e-9249-4060-afc4-55140bcb9911") : object "kube-system"/"original-pull-secret" not registered
Jul 10 22:55:48.186742 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:48.186525    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs podName:9829a27f-3dbd-44c7-b278-f26beb69f82a nodeName:}" failed. No retries permitted until 2026-07-10 22:55:56.186514063 +0000 UTC m=+17.073250186 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs") pod "network-metrics-daemon-w9kxw" (UID: "9829a27f-3dbd-44c7-b278-f26beb69f82a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 22:55:48.287344 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:48.287299    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8924\" (UniqueName: \"kubernetes.io/projected/3f0e1c5a-a319-437d-9aef-8b3c4150a3b1-kube-api-access-j8924\") pod \"network-check-target-8nt9g\" (UID: \"3f0e1c5a-a319-437d-9aef-8b3c4150a3b1\") " pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:55:48.287538 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:48.287518    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 22:55:48.287591 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:48.287543    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 22:55:48.287591 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:48.287556    2563 projected.go:196] Error preparing data for projected volume kube-api-access-j8924 for pod openshift-network-diagnostics/network-check-target-8nt9g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 22:55:48.287690 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:48.287620    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3f0e1c5a-a319-437d-9aef-8b3c4150a3b1-kube-api-access-j8924 podName:3f0e1c5a-a319-437d-9aef-8b3c4150a3b1 nodeName:}" failed. No retries permitted until 2026-07-10 22:55:56.287601028 +0000 UTC m=+17.174337172 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-j8924" (UniqueName: "kubernetes.io/projected/3f0e1c5a-a319-437d-9aef-8b3c4150a3b1-kube-api-access-j8924") pod "network-check-target-8nt9g" (UID: "3f0e1c5a-a319-437d-9aef-8b3c4150a3b1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 22:55:48.550444 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:48.550270    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:55:48.550444 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:48.550305    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:55:48.550658 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:48.550578    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zb67" podUID="3734910e-9249-4060-afc4-55140bcb9911"
Jul 10 22:55:48.550658 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:48.550609    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9kxw" podUID="9829a27f-3dbd-44c7-b278-f26beb69f82a"
Jul 10 22:55:49.551336 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:49.550940    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:55:49.551336 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:49.551042    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nt9g" podUID="3f0e1c5a-a319-437d-9aef-8b3c4150a3b1"
Jul 10 22:55:50.550571 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:50.550323    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:55:50.550741 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:50.550669    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9kxw" podUID="9829a27f-3dbd-44c7-b278-f26beb69f82a"
Jul 10 22:55:50.558135 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:50.558090    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:55:50.558662 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:50.558314    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zb67" podUID="3734910e-9249-4060-afc4-55140bcb9911"
Jul 10 22:55:51.550740 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:51.550706    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:55:51.550917 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:51.550828    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nt9g" podUID="3f0e1c5a-a319-437d-9aef-8b3c4150a3b1"
Jul 10 22:55:52.549996 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:52.549962    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:55:52.549996 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:52.550004    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:55:52.550420 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:52.550096    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9kxw" podUID="9829a27f-3dbd-44c7-b278-f26beb69f82a"
Jul 10 22:55:52.550420 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:52.550215    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zb67" podUID="3734910e-9249-4060-afc4-55140bcb9911"
Jul 10 22:55:53.550537 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:53.550506    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:55:53.551042 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:53.550639    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nt9g" podUID="3f0e1c5a-a319-437d-9aef-8b3c4150a3b1"
Jul 10 22:55:54.549872 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:54.549833    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:55:54.550069 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:54.549988    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9kxw" podUID="9829a27f-3dbd-44c7-b278-f26beb69f82a"
Jul 10 22:55:54.550069 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:54.550049    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:55:54.550209 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:54.550185    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zb67" podUID="3734910e-9249-4060-afc4-55140bcb9911"
Jul 10 22:55:55.550305 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:55.550271    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:55:55.550817 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:55.550422    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nt9g" podUID="3f0e1c5a-a319-437d-9aef-8b3c4150a3b1"
Jul 10 22:55:56.248529 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:56.248484    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3734910e-9249-4060-afc4-55140bcb9911-original-pull-secret\") pod \"global-pull-secret-syncer-7zb67\" (UID: \"3734910e-9249-4060-afc4-55140bcb9911\") " pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:55:56.248529 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:56.248538    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs\") pod \"network-metrics-daemon-w9kxw\" (UID: \"9829a27f-3dbd-44c7-b278-f26beb69f82a\") " pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:55:56.248741 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:56.248649    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 22:55:56.248741 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:56.248678    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 22:55:56.248741 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:56.248733    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3734910e-9249-4060-afc4-55140bcb9911-original-pull-secret podName:3734910e-9249-4060-afc4-55140bcb9911 nodeName:}" failed. No retries permitted until 2026-07-10 22:56:12.248701635 +0000 UTC m=+33.135437756 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3734910e-9249-4060-afc4-55140bcb9911-original-pull-secret") pod "global-pull-secret-syncer-7zb67" (UID: "3734910e-9249-4060-afc4-55140bcb9911") : object "kube-system"/"original-pull-secret" not registered
Jul 10 22:55:56.248865 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:56.248749    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs podName:9829a27f-3dbd-44c7-b278-f26beb69f82a nodeName:}" failed. No retries permitted until 2026-07-10 22:56:12.248741749 +0000 UTC m=+33.135477870 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs") pod "network-metrics-daemon-w9kxw" (UID: "9829a27f-3dbd-44c7-b278-f26beb69f82a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 22:55:56.349452 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:56.349413    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8924\" (UniqueName: \"kubernetes.io/projected/3f0e1c5a-a319-437d-9aef-8b3c4150a3b1-kube-api-access-j8924\") pod \"network-check-target-8nt9g\" (UID: \"3f0e1c5a-a319-437d-9aef-8b3c4150a3b1\") " pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:55:56.349638 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:56.349604    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 22:55:56.349638 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:56.349631    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 22:55:56.349729 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:56.349645    2563 projected.go:196] Error preparing data for projected volume kube-api-access-j8924 for pod openshift-network-diagnostics/network-check-target-8nt9g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 22:55:56.349729 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:56.349711    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3f0e1c5a-a319-437d-9aef-8b3c4150a3b1-kube-api-access-j8924 podName:3f0e1c5a-a319-437d-9aef-8b3c4150a3b1 nodeName:}" failed. No retries permitted until 2026-07-10 22:56:12.349692824 +0000 UTC m=+33.236428949 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-j8924" (UniqueName: "kubernetes.io/projected/3f0e1c5a-a319-437d-9aef-8b3c4150a3b1-kube-api-access-j8924") pod "network-check-target-8nt9g" (UID: "3f0e1c5a-a319-437d-9aef-8b3c4150a3b1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 22:55:56.550424 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:56.550323    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:55:56.550424 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:56.550385    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:55:56.550916 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:56.550477    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9kxw" podUID="9829a27f-3dbd-44c7-b278-f26beb69f82a"
Jul 10 22:55:56.550916 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:56.550603    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zb67" podUID="3734910e-9249-4060-afc4-55140bcb9911"
Jul 10 22:55:57.550166 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:57.550127    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:55:57.550415 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:57.550266    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nt9g" podUID="3f0e1c5a-a319-437d-9aef-8b3c4150a3b1"
Jul 10 22:55:58.550623 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:58.550568    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:55:58.551080 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:58.550626    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:55:58.551080 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:58.550702    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zb67" podUID="3734910e-9249-4060-afc4-55140bcb9911"
Jul 10 22:55:58.551080 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:58.550850    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9kxw" podUID="9829a27f-3dbd-44c7-b278-f26beb69f82a"
Jul 10 22:55:59.551103 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:55:59.551068    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:55:59.551525 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:55:59.551167    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nt9g" podUID="3f0e1c5a-a319-437d-9aef-8b3c4150a3b1"
Jul 10 22:56:00.550280 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:00.550149    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:56:00.550469 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:00.550281    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9kxw" podUID="9829a27f-3dbd-44c7-b278-f26beb69f82a"
Jul 10 22:56:00.550469 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:00.550149    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:56:00.550469 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:00.550390    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zb67" podUID="3734910e-9249-4060-afc4-55140bcb9911"
Jul 10 22:56:01.550721 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:01.550683    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:56:01.551084 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:01.550825    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nt9g" podUID="3f0e1c5a-a319-437d-9aef-8b3c4150a3b1"
Jul 10 22:56:02.550858 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:02.550561    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:56:02.551462 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:02.550615    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:56:02.551462 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:02.550932    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zb67" podUID="3734910e-9249-4060-afc4-55140bcb9911"
Jul 10 22:56:02.551462 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:02.551057    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9kxw" podUID="9829a27f-3dbd-44c7-b278-f26beb69f82a"
Jul 10 22:56:02.786614 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:02.786576    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rdxmp" event={"ID":"b8382cd0-6bcd-4037-814a-dfa9c652d3bd","Type":"ContainerStarted","Data":"9d6ff7a0682f9e4beadf019c9bef19f16c855efae3a06935477ff0e1672bdd4b"}
Jul 10 22:56:02.788670 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:02.788629    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks" event={"ID":"257b0f09-db72-43e3-a3d5-a28a4fcdc3d6","Type":"ContainerStarted","Data":"ae681643cd0b9687d77811acbfa5617bfab02653f19d6bb1a871ada8c3a473bd"}
Jul 10 22:56:02.797587 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:02.797561    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/ovn-acl-logging/0.log"
Jul 10 22:56:02.797984 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:02.797958    2563 generic.go:356] "Generic (PLEG): container finished" podID="861ff3e2-b6a3-4a8b-a23f-88faf0b7e203" containerID="99cff1c5fad69983c666cb21acc6f618ffc494b3f649fc55cbc4803dd1689153" exitCode=1
Jul 10 22:56:02.798193 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:02.798029    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qj86t" event={"ID":"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203","Type":"ContainerStarted","Data":"f27490ed9c27062547e156b07ca6a9577843338ec88c7359dc1c23c304fc15f6"}
Jul 10 22:56:02.798426 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:02.798395    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qj86t" event={"ID":"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203","Type":"ContainerStarted","Data":"d63b6bc89517298b301aece18c39ba87e1d3920f7d2f543741b601119e00c889"}
Jul 10 22:56:02.798503 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:02.798445    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qj86t" event={"ID":"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203","Type":"ContainerStarted","Data":"89df76f1cda01efd442c18a1247e0e69797cd413df9898dcc7c714cc94651e9a"}
Jul 10 22:56:02.798503 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:02.798474    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qj86t" event={"ID":"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203","Type":"ContainerStarted","Data":"e13436bb121e01015f64ea8c610f0cf7492ea48abcd9b73f39d48ee6e7840356"}
Jul 10 22:56:02.798570 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:02.798515    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qj86t" event={"ID":"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203","Type":"ContainerDied","Data":"99cff1c5fad69983c666cb21acc6f618ffc494b3f649fc55cbc4803dd1689153"}
Jul 10 22:56:02.798570 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:02.798545    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qj86t" event={"ID":"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203","Type":"ContainerStarted","Data":"fe52a7f1777e4b4393ae52e82a083fb9948cfc4c5b298ce2dda37bb7b88716ba"}
Jul 10 22:56:02.800225 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:02.800188    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kd62p" event={"ID":"54d84e33-5ba4-4b6c-8c8f-4ddaac24eb03","Type":"ContainerStarted","Data":"af211a93cbc2ae02cfb98ba954d95f6bd51268574a44aa29b761308883429eec"}
Jul 10 22:56:02.805560 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:02.805525    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-w2wjh" event={"ID":"50a4a720-01af-4105-9de4-260ee26283a4","Type":"ContainerStarted","Data":"846a0b91cd6ccd3bd027f2a942d507007dd564ca434411c1141d1c8703e64605"}
Jul 10 22:56:02.806420 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:02.806318    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rdxmp" podStartSLOduration=2.9807220599999997 podStartE2EDuration="23.806302756s" totalImagesPullingTime="20.825580696s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 22:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:56:02.805983528 +0000 UTC m=+23.692719696" watchObservedRunningTime="2026-07-10 22:56:02.806302756 +0000 UTC m=+23.693038897"
Jul 10 22:56:02.807199 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:02.807180    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-298xj" event={"ID":"e17cacf8-0106-484f-a226-2f584a8fc5a5","Type":"ContainerStarted","Data":"94fd837ef8429812baab6e7e324ba31c4f61806fba86589cdc2f8f317ae6fc21"}
Jul 10 22:56:02.810110 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:02.810081    2563 generic.go:356] "Generic (PLEG): container finished" podID="98bd729d123769a48c9d58a5f9915bea" containerID="b9e3442999f55d13a326843336231d47d8bfbc3221fe36ac6677c54c43e527e9" exitCode=0
Jul 10 22:56:02.810234 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:02.810176    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-239.ec2.internal" event={"ID":"98bd729d123769a48c9d58a5f9915bea","Type":"ContainerDied","Data":"b9e3442999f55d13a326843336231d47d8bfbc3221fe36ac6677c54c43e527e9"}
Jul 10 22:56:02.811869 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:02.811843    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fwk45" event={"ID":"0ee6c937-5bad-4104-916a-71224add52b5","Type":"ContainerStarted","Data":"0c6b8e2fe9eb150db59b2545afc628cd45d0263dfe5b0e5505baf53ce6d14133"}
Jul 10 22:56:02.813648 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:02.813623    2563 generic.go:356] "Generic (PLEG): container finished" podID="cdbc917d-2667-438e-928b-326933a924d8" containerID="159af61e5ca39d8c035401fcbc61bdbbc80d829c76a4a71b97aacad791e42c14" exitCode=0
Jul 10 22:56:02.813763 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:02.813691    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xpkzr" event={"ID":"cdbc917d-2667-438e-928b-326933a924d8","Type":"ContainerDied","Data":"159af61e5ca39d8c035401fcbc61bdbbc80d829c76a4a71b97aacad791e42c14"}
Jul 10 22:56:02.815606 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:02.815551    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-239.ec2.internal" event={"ID":"6c99c1f4b605d867ddb0d2d31d3a10b5","Type":"ContainerStarted","Data":"80ee0c24487315ac31722559823f491a9edcb6cb3e59e8968e6b5dd498b42894"}
Jul 10 22:56:02.823007 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:02.822950    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-w2wjh" podStartSLOduration=2.889033254 podStartE2EDuration="23.822933049s" totalImagesPullingTime="20.933899795s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 22:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:56:02.822762161 +0000 UTC m=+23.709498304" watchObservedRunningTime="2026-07-10 22:56:02.822933049 +0000 UTC m=+23.709669197"
Jul 10 22:56:02.857956 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:02.857892    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kd62p" podStartSLOduration=7.734954044 podStartE2EDuration="23.857872445s" totalImagesPullingTime="16.122918401s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 22:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:56:02.835616916 +0000 UTC m=+23.722353060" watchObservedRunningTime="2026-07-10 22:56:02.857872445 +0000 UTC m=+23.744608587"
Jul 10 22:56:02.880086 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:02.880013    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-fwk45" podStartSLOduration=3.09355378 podStartE2EDuration="23.879996789s" totalImagesPullingTime="20.786443009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:56:02.8587389 +0000 UTC m=+23.745475045" watchObservedRunningTime="2026-07-10 22:56:02.879996789 +0000 UTC m=+23.766732983"
Jul 10 22:56:02.899557 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:02.899506    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-239.ec2.internal" podStartSLOduration=22.899489755 podStartE2EDuration="22.899489755s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 22:55:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:56:02.898977389 +0000 UTC m=+23.785713532" watchObservedRunningTime="2026-07-10 22:56:02.899489755 +0000 UTC m=+23.786225899"
Jul 10 22:56:02.985472 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:02.985345    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-298xj" podStartSLOduration=3.062252661 podStartE2EDuration="23.985327228s" totalImagesPullingTime="20.923074567s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 22:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:56:02.915212619 +0000 UTC m=+23.801948761" watchObservedRunningTime="2026-07-10 22:56:02.985327228 +0000 UTC m=+23.872063382"
Jul 10 22:56:03.550495 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:03.550468    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:56:03.550638 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:03.550595    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nt9g" podUID="3f0e1c5a-a319-437d-9aef-8b3c4150a3b1"
Jul 10 22:56:03.624457 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:03.624429    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 10 22:56:03.820158 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:03.820076    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks" event={"ID":"257b0f09-db72-43e3-a3d5-a28a4fcdc3d6","Type":"ContainerStarted","Data":"f47a765863abbf6fa20e0529cb9984d83ba0dc18dd6a34537b961ab3fcc4b88e"}
Jul 10 22:56:03.822285 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:03.822248    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-239.ec2.internal" event={"ID":"98bd729d123769a48c9d58a5f9915bea","Type":"ContainerStarted","Data":"a09bb23c095cbf3c52ed1ee64974702377abca9320b06ef1150c6e8e738c783a"}
Jul 10 22:56:03.824041 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:03.824013    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hvkfr" event={"ID":"3fe0a8af-e7c7-47cf-81d5-5e0c3c0ad4e2","Type":"ContainerStarted","Data":"ef0dfd0f24dc5068c9cc0b672a100c39246d151a8311bae9e8216f762e0ef424"}
Jul 10 22:56:03.838222 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:03.838157    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-239.ec2.internal" podStartSLOduration=23.838135613 podStartE2EDuration="23.838135613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 22:55:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:56:03.837702471 +0000 UTC m=+24.724438616" watchObservedRunningTime="2026-07-10 22:56:03.838135613 +0000 UTC m=+24.724871758"
Jul 10 22:56:03.852632 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:03.852574    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hvkfr" podStartSLOduration=3.993097839 podStartE2EDuration="24.852558681s" totalImagesPullingTime="20.859460842s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 22:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:56:03.851985467 +0000 UTC m=+24.738721611" watchObservedRunningTime="2026-07-10 22:56:03.852558681 +0000 UTC m=+24.739294823"
Jul 10 22:56:04.494408 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:04.494276    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T22:56:03.624449904Z","UUID":"59514a84-88ae-46ef-aa45-6b6454f8a9dd","Handler":null,"Name":"","Endpoint":""}
Jul 10 22:56:04.496058 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:04.496033    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 10 22:56:04.496183 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:04.496084    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 10 22:56:04.550485 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:04.550447    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:56:04.550485 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:04.550473    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:56:04.550709 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:04.550571    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zb67" podUID="3734910e-9249-4060-afc4-55140bcb9911"
Jul 10 22:56:04.550709 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:04.550701    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9kxw" podUID="9829a27f-3dbd-44c7-b278-f26beb69f82a"
Jul 10 22:56:05.550325 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:05.550281    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:56:05.550869 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:05.550454    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nt9g" podUID="3f0e1c5a-a319-437d-9aef-8b3c4150a3b1"
Jul 10 22:56:05.830492 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:05.830406    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks" event={"ID":"257b0f09-db72-43e3-a3d5-a28a4fcdc3d6","Type":"ContainerStarted","Data":"55304a4d58e50af65203b83082fe2a7a57ea17f88801996b015759a528da95a7"}
Jul 10 22:56:05.834000 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:05.833978    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/ovn-acl-logging/0.log"
Jul 10 22:56:05.834458 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:05.834431    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qj86t" event={"ID":"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203","Type":"ContainerStarted","Data":"0d729a526686c1918cd90cecb982b5f45db754af8f3d6c8896f7f15c4408e1b7"}
Jul 10 22:56:05.849566 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:05.849514    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xbkks" podStartSLOduration=3.779318994 podStartE2EDuration="26.849491742s" totalImagesPullingTime="23.070172748s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 22:55:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:56:05.848492876 +0000 UTC m=+26.735229021" watchObservedRunningTime="2026-07-10 22:56:05.849491742 +0000 UTC m=+26.736227875"
Jul 10 22:56:06.550508 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:06.550280    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:56:06.550943 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:06.550336    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:56:06.550943 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:06.550624    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zb67" podUID="3734910e-9249-4060-afc4-55140bcb9911"
Jul 10 22:56:06.550943 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:06.550740    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9kxw" podUID="9829a27f-3dbd-44c7-b278-f26beb69f82a"
Jul 10 22:56:07.225293 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:07.225200    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-fwk45"
Jul 10 22:56:07.225969 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:07.225943    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-fwk45"
Jul 10 22:56:07.550116 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:07.550079    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:56:07.550258 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:07.550228    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nt9g" podUID="3f0e1c5a-a319-437d-9aef-8b3c4150a3b1"
Jul 10 22:56:07.843235 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:07.843071    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/ovn-acl-logging/0.log"
Jul 10 22:56:07.844076 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:07.843756    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qj86t" event={"ID":"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203","Type":"ContainerStarted","Data":"050b5330e384f5bb81ecd118b895db0356622bbaef3b4ee93aa9f0272c7e02ff"}
Jul 10 22:56:07.844158 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:07.844127    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:56:07.844215 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:07.844202    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:56:07.844215 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:07.844210    2563 scope.go:117] "RemoveContainer" containerID="99cff1c5fad69983c666cb21acc6f618ffc494b3f649fc55cbc4803dd1689153"
Jul 10 22:56:07.846141 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:07.846112    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xpkzr" event={"ID":"cdbc917d-2667-438e-928b-326933a924d8","Type":"ContainerStarted","Data":"0e0d71facdc60f1b20e3a425b1c8dbc6fe3ba8fd1fb3dac2d88556836abb5af5"}
Jul 10 22:56:07.846777 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:07.846555    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-fwk45"
Jul 10 22:56:07.847082 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:07.847043    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-fwk45"
Jul 10 22:56:07.860970 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:07.860949    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:56:08.550503 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:08.550463    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:56:08.550670 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:08.550509    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:56:08.550670 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:08.550577    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zb67" podUID="3734910e-9249-4060-afc4-55140bcb9911"
Jul 10 22:56:08.550751 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:08.550692    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9kxw" podUID="9829a27f-3dbd-44c7-b278-f26beb69f82a"
Jul 10 22:56:08.851692 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:08.851609    2563 generic.go:356] "Generic (PLEG): container finished" podID="cdbc917d-2667-438e-928b-326933a924d8" containerID="0e0d71facdc60f1b20e3a425b1c8dbc6fe3ba8fd1fb3dac2d88556836abb5af5" exitCode=0
Jul 10 22:56:08.852221 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:08.851715    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xpkzr" event={"ID":"cdbc917d-2667-438e-928b-326933a924d8","Type":"ContainerDied","Data":"0e0d71facdc60f1b20e3a425b1c8dbc6fe3ba8fd1fb3dac2d88556836abb5af5"}
Jul 10 22:56:08.856307 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:08.856290    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/ovn-acl-logging/0.log"
Jul 10 22:56:08.856850 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:08.856803    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qj86t" event={"ID":"861ff3e2-b6a3-4a8b-a23f-88faf0b7e203","Type":"ContainerStarted","Data":"4a4a3fc1b35c181163c03641cf66b6547f6b8082803bcc571803d7d903e2f3a1"}
Jul 10 22:56:08.857654 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:08.857120    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:56:08.880418 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:08.880382    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:56:08.909678 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:08.909618    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qj86t" podStartSLOduration=8.885940544 podStartE2EDuration="29.90959921s" totalImagesPullingTime="21.023658666s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 22:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:56:08.908625112 +0000 UTC m=+29.795361267" watchObservedRunningTime="2026-07-10 22:56:08.90959921 +0000 UTC m=+29.796335354"
Jul 10 22:56:09.556909 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:09.556877    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:56:09.557090 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:09.556986    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nt9g" podUID="3f0e1c5a-a319-437d-9aef-8b3c4150a3b1"
Jul 10 22:56:09.805068 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:09.805038    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7zb67"]
Jul 10 22:56:09.805245 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:09.805174    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:56:09.805306 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:09.805285    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zb67" podUID="3734910e-9249-4060-afc4-55140bcb9911"
Jul 10 22:56:09.808605 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:09.808513    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w9kxw"]
Jul 10 22:56:09.808745 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:09.808652    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:56:09.808813 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:09.808765    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9kxw" podUID="9829a27f-3dbd-44c7-b278-f26beb69f82a"
Jul 10 22:56:09.809253 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:09.809230    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8nt9g"]
Jul 10 22:56:09.859226 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:09.859191    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:56:09.859890 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:09.859859    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nt9g" podUID="3f0e1c5a-a319-437d-9aef-8b3c4150a3b1"
Jul 10 22:56:10.864112 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:10.864076    2563 generic.go:356] "Generic (PLEG): container finished" podID="cdbc917d-2667-438e-928b-326933a924d8" containerID="b88dac2fbea3276b3a47f508ee26b674724d81e466fece6dc34aebdf35bb0e2d" exitCode=0
Jul 10 22:56:10.864607 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:10.864156    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xpkzr" event={"ID":"cdbc917d-2667-438e-928b-326933a924d8","Type":"ContainerDied","Data":"b88dac2fbea3276b3a47f508ee26b674724d81e466fece6dc34aebdf35bb0e2d"}
Jul 10 22:56:11.554760 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:11.554726    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:56:11.554926 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:11.554775    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:56:11.554926 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:11.554850    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9kxw" podUID="9829a27f-3dbd-44c7-b278-f26beb69f82a"
Jul 10 22:56:11.554926 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:11.554903    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zb67" podUID="3734910e-9249-4060-afc4-55140bcb9911"
Jul 10 22:56:11.555027 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:11.554946    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:56:11.555027 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:11.555000    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nt9g" podUID="3f0e1c5a-a319-437d-9aef-8b3c4150a3b1"
Jul 10 22:56:11.872196 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:11.872024    2563 generic.go:356] "Generic (PLEG): container finished" podID="cdbc917d-2667-438e-928b-326933a924d8" containerID="628f5f0643d613334b52f8815e490cf80de60ff580f845189cce76d9f49a2dd8" exitCode=0
Jul 10 22:56:11.872196 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:11.872089    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xpkzr" event={"ID":"cdbc917d-2667-438e-928b-326933a924d8","Type":"ContainerDied","Data":"628f5f0643d613334b52f8815e490cf80de60ff580f845189cce76d9f49a2dd8"}
Jul 10 22:56:12.267345 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:12.267305    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3734910e-9249-4060-afc4-55140bcb9911-original-pull-secret\") pod \"global-pull-secret-syncer-7zb67\" (UID: \"3734910e-9249-4060-afc4-55140bcb9911\") " pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:56:12.267345 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:12.267344    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs\") pod \"network-metrics-daemon-w9kxw\" (UID: \"9829a27f-3dbd-44c7-b278-f26beb69f82a\") " pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:56:12.267579 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:12.267453    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 22:56:12.267579 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:12.267454    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 22:56:12.267579 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:12.267505    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3734910e-9249-4060-afc4-55140bcb9911-original-pull-secret podName:3734910e-9249-4060-afc4-55140bcb9911 nodeName:}" failed. No retries permitted until 2026-07-10 22:56:44.267491272 +0000 UTC m=+65.154227393 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3734910e-9249-4060-afc4-55140bcb9911-original-pull-secret") pod "global-pull-secret-syncer-7zb67" (UID: "3734910e-9249-4060-afc4-55140bcb9911") : object "kube-system"/"original-pull-secret" not registered
Jul 10 22:56:12.267579 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:12.267520    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs podName:9829a27f-3dbd-44c7-b278-f26beb69f82a nodeName:}" failed. No retries permitted until 2026-07-10 22:56:44.267513221 +0000 UTC m=+65.154249341 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs") pod "network-metrics-daemon-w9kxw" (UID: "9829a27f-3dbd-44c7-b278-f26beb69f82a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 22:56:12.368544 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:12.368506    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8924\" (UniqueName: \"kubernetes.io/projected/3f0e1c5a-a319-437d-9aef-8b3c4150a3b1-kube-api-access-j8924\") pod \"network-check-target-8nt9g\" (UID: \"3f0e1c5a-a319-437d-9aef-8b3c4150a3b1\") " pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:56:12.368714 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:12.368663    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 22:56:12.368714 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:12.368684    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 22:56:12.368714 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:12.368693    2563 projected.go:196] Error preparing data for projected volume kube-api-access-j8924 for pod openshift-network-diagnostics/network-check-target-8nt9g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 22:56:12.368835 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:12.368751    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3f0e1c5a-a319-437d-9aef-8b3c4150a3b1-kube-api-access-j8924 podName:3f0e1c5a-a319-437d-9aef-8b3c4150a3b1 nodeName:}" failed. No retries permitted until 2026-07-10 22:56:44.368737834 +0000 UTC m=+65.255473959 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-j8924" (UniqueName: "kubernetes.io/projected/3f0e1c5a-a319-437d-9aef-8b3c4150a3b1-kube-api-access-j8924") pod "network-check-target-8nt9g" (UID: "3f0e1c5a-a319-437d-9aef-8b3c4150a3b1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 22:56:13.550657 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:13.550567    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:56:13.551034 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:13.550564    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:56:13.551034 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:13.550691    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w9kxw" podUID="9829a27f-3dbd-44c7-b278-f26beb69f82a"
Jul 10 22:56:13.551034 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:13.550701    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:56:13.551034 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:13.550784    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nt9g" podUID="3f0e1c5a-a319-437d-9aef-8b3c4150a3b1"
Jul 10 22:56:13.551034 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:13.550880    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7zb67" podUID="3734910e-9249-4060-afc4-55140bcb9911"
Jul 10 22:56:14.503469 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.503431    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-239.ec2.internal" event="NodeReady"
Jul 10 22:56:14.503646 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.503579    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 10 22:56:14.534240 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.534210    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-l9t9h"]
Jul 10 22:56:14.538611 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.538529    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-769cc77955-d2747"]
Jul 10 22:56:14.538745 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.538659    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-l9t9h"
Jul 10 22:56:14.540804 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.540752    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 10 22:56:14.540940 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.540924    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-xxpgj\""
Jul 10 22:56:14.541168 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.541145    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 10 22:56:14.542502 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.542480    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:14.544973 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.544337    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 10 22:56:14.544973 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.544517    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 10 22:56:14.544973 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.544801    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 10 22:56:14.544973 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.544950    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-67lgp\""
Jul 10 22:56:14.550039 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.549925    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-l9t9h"]
Jul 10 22:56:14.551846 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.551808    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-769cc77955-d2747"]
Jul 10 22:56:14.554981 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.554952    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-9ttw7"]
Jul 10 22:56:14.559165 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.559136    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 10 22:56:14.559452 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.559354    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-v2p2h"]
Jul 10 22:56:14.559589 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.559542    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9ttw7"
Jul 10 22:56:14.562121 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.562099    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 10 22:56:14.562219 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.562192    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 10 22:56:14.562355 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.562338    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ld9cp\""
Jul 10 22:56:14.562587 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.562568    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 10 22:56:14.563685 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.563666    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v2p2h"
Jul 10 22:56:14.565940 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.565919    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 10 22:56:14.565940 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.565929    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-j5n6l\""
Jul 10 22:56:14.566238 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.566220    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 10 22:56:14.567419 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.567400    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9ttw7"]
Jul 10 22:56:14.572757 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.572734    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-v2p2h"]
Jul 10 22:56:14.679278 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.679232    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:14.685299 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.685262    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e7f0af82-786b-4b48-be84-9ced1047ff52-image-registry-private-configuration\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:14.685482 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.685310    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/741d147a-735b-40ef-a55b-8e8f03b062d0-config-volume\") pod \"dns-default-v2p2h\" (UID: \"741d147a-735b-40ef-a55b-8e8f03b062d0\") " pod="openshift-dns/dns-default-v2p2h"
Jul 10 22:56:14.685482 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.685339    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2981ee03-5541-484c-8226-2d606103d31e-nginx-conf\") pod \"networking-console-plugin-6cf856758c-l9t9h\" (UID: \"2981ee03-5541-484c-8226-2d606103d31e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-l9t9h"
Jul 10 22:56:14.685482 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.685380    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/741d147a-735b-40ef-a55b-8e8f03b062d0-tmp-dir\") pod \"dns-default-v2p2h\" (UID: \"741d147a-735b-40ef-a55b-8e8f03b062d0\") " pod="openshift-dns/dns-default-v2p2h"
Jul 10 22:56:14.685482 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.685400    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbjj6\" (UniqueName: \"kubernetes.io/projected/741d147a-735b-40ef-a55b-8e8f03b062d0-kube-api-access-nbjj6\") pod \"dns-default-v2p2h\" (UID: \"741d147a-735b-40ef-a55b-8e8f03b062d0\") " pod="openshift-dns/dns-default-v2p2h"
Jul 10 22:56:14.685660 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.685494    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/67dc0695-fad8-4ecf-bc13-3a7410c1b195-cert\") pod \"ingress-canary-9ttw7\" (UID: \"67dc0695-fad8-4ecf-bc13-3a7410c1b195\") " pod="openshift-ingress-canary/ingress-canary-9ttw7"
Jul 10 22:56:14.685660 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.685540    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-certificates\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:14.685660 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.685592    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e7f0af82-786b-4b48-be84-9ced1047ff52-installation-pull-secrets\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:14.685803 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.685668    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e7f0af82-786b-4b48-be84-9ced1047ff52-ca-trust-extracted\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:14.685803 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.685711    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kb2l\" (UniqueName: \"kubernetes.io/projected/67dc0695-fad8-4ecf-bc13-3a7410c1b195-kube-api-access-4kb2l\") pod \"ingress-canary-9ttw7\" (UID: \"67dc0695-fad8-4ecf-bc13-3a7410c1b195\") " pod="openshift-ingress-canary/ingress-canary-9ttw7"
Jul 10 22:56:14.685803 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.685747    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls\") pod \"dns-default-v2p2h\" (UID: \"741d147a-735b-40ef-a55b-8e8f03b062d0\") " pod="openshift-dns/dns-default-v2p2h"
Jul 10 22:56:14.685803 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.685785    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e7f0af82-786b-4b48-be84-9ced1047ff52-trusted-ca\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:14.685984 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.685807    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-bound-sa-token\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:14.685984 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.685869    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-l9t9h\" (UID: \"2981ee03-5541-484c-8226-2d606103d31e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-l9t9h"
Jul 10 22:56:14.685984 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.685945    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftm6k\" (UniqueName: \"kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-kube-api-access-ftm6k\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:14.685984 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.685980    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:14.782114 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.782024    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:14.787018 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.786974    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/67dc0695-fad8-4ecf-bc13-3a7410c1b195-cert\") pod \"ingress-canary-9ttw7\" (UID: \"67dc0695-fad8-4ecf-bc13-3a7410c1b195\") " pod="openshift-ingress-canary/ingress-canary-9ttw7"
Jul 10 22:56:14.787018 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.787023    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-certificates\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:14.787245 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.787046    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e7f0af82-786b-4b48-be84-9ced1047ff52-installation-pull-secrets\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:14.787245 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.787074    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e7f0af82-786b-4b48-be84-9ced1047ff52-ca-trust-extracted\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:14.787245 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.787098    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kb2l\" (UniqueName: \"kubernetes.io/projected/67dc0695-fad8-4ecf-bc13-3a7410c1b195-kube-api-access-4kb2l\") pod \"ingress-canary-9ttw7\" (UID: \"67dc0695-fad8-4ecf-bc13-3a7410c1b195\") " pod="openshift-ingress-canary/ingress-canary-9ttw7"
Jul 10 22:56:14.787245 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.787122    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls\") pod \"dns-default-v2p2h\" (UID: \"741d147a-735b-40ef-a55b-8e8f03b062d0\") " pod="openshift-dns/dns-default-v2p2h"
Jul 10 22:56:14.787245 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.787155    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e7f0af82-786b-4b48-be84-9ced1047ff52-trusted-ca\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:14.787245 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.787176    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-bound-sa-token\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:14.787245 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.787198    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-l9t9h\" (UID: \"2981ee03-5541-484c-8226-2d606103d31e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-l9t9h"
Jul 10 22:56:14.787571 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.787254    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftm6k\" (UniqueName: \"kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-kube-api-access-ftm6k\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:14.787571 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.787280    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:14.787571 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.787317    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e7f0af82-786b-4b48-be84-9ced1047ff52-image-registry-private-configuration\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:14.787571 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.787348    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/741d147a-735b-40ef-a55b-8e8f03b062d0-config-volume\") pod \"dns-default-v2p2h\" (UID: \"741d147a-735b-40ef-a55b-8e8f03b062d0\") " pod="openshift-dns/dns-default-v2p2h"
Jul 10 22:56:14.787571 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.787399    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2981ee03-5541-484c-8226-2d606103d31e-nginx-conf\") pod \"networking-console-plugin-6cf856758c-l9t9h\" (UID: \"2981ee03-5541-484c-8226-2d606103d31e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-l9t9h"
Jul 10 22:56:14.787571 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.787424    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/741d147a-735b-40ef-a55b-8e8f03b062d0-tmp-dir\") pod \"dns-default-v2p2h\" (UID: \"741d147a-735b-40ef-a55b-8e8f03b062d0\") " pod="openshift-dns/dns-default-v2p2h"
Jul 10 22:56:14.787571 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.787448    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbjj6\" (UniqueName: \"kubernetes.io/projected/741d147a-735b-40ef-a55b-8e8f03b062d0-kube-api-access-nbjj6\") pod \"dns-default-v2p2h\" (UID: \"741d147a-735b-40ef-a55b-8e8f03b062d0\") " pod="openshift-dns/dns-default-v2p2h"
Jul 10 22:56:14.787965 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:14.787941    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 22:56:14.788082 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:14.788054    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert podName:2981ee03-5541-484c-8226-2d606103d31e nodeName:}" failed. No retries permitted until 2026-07-10 22:56:15.288036629 +0000 UTC m=+36.174772754 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-l9t9h" (UID: "2981ee03-5541-484c-8226-2d606103d31e") : secret "networking-console-plugin-cert" not found
Jul 10 22:56:14.788413 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:14.788389    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 22:56:14.788413 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:14.788413    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-769cc77955-d2747: secret "image-registry-tls" not found
Jul 10 22:56:14.788573 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:14.788469    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls podName:e7f0af82-786b-4b48-be84-9ced1047ff52 nodeName:}" failed. No retries permitted until 2026-07-10 22:56:15.288454115 +0000 UTC m=+36.175190237 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls") pod "image-registry-769cc77955-d2747" (UID: "e7f0af82-786b-4b48-be84-9ced1047ff52") : secret "image-registry-tls" not found
Jul 10 22:56:14.788573 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:14.788470    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 22:56:14.788573 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.788484    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-certificates\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:14.788573 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:14.788506    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls podName:741d147a-735b-40ef-a55b-8e8f03b062d0 nodeName:}" failed. No retries permitted until 2026-07-10 22:56:15.288497093 +0000 UTC m=+36.175233220 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls") pod "dns-default-v2p2h" (UID: "741d147a-735b-40ef-a55b-8e8f03b062d0") : secret "dns-default-metrics-tls" not found
Jul 10 22:56:14.788775 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.788728    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e7f0af82-786b-4b48-be84-9ced1047ff52-ca-trust-extracted\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:14.789027 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.789002    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/741d147a-735b-40ef-a55b-8e8f03b062d0-tmp-dir\") pod \"dns-default-v2p2h\" (UID: \"741d147a-735b-40ef-a55b-8e8f03b062d0\") " pod="openshift-dns/dns-default-v2p2h"
Jul 10 22:56:14.789446 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.789423    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e7f0af82-786b-4b48-be84-9ced1047ff52-trusted-ca\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:14.789544 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.789447    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2981ee03-5541-484c-8226-2d606103d31e-nginx-conf\") pod \"networking-console-plugin-6cf856758c-l9t9h\" (UID: \"2981ee03-5541-484c-8226-2d606103d31e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-l9t9h"
Jul 10 22:56:14.789544 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.789502    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/741d147a-735b-40ef-a55b-8e8f03b062d0-config-volume\") pod \"dns-default-v2p2h\" (UID: \"741d147a-735b-40ef-a55b-8e8f03b062d0\") " pod="openshift-dns/dns-default-v2p2h"
Jul 10 22:56:14.792777 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.792753    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/67dc0695-fad8-4ecf-bc13-3a7410c1b195-cert\") pod \"ingress-canary-9ttw7\" (UID: \"67dc0695-fad8-4ecf-bc13-3a7410c1b195\") " pod="openshift-ingress-canary/ingress-canary-9ttw7"
Jul 10 22:56:14.792777 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.792760    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e7f0af82-786b-4b48-be84-9ced1047ff52-image-registry-private-configuration\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:14.792934 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.792760    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e7f0af82-786b-4b48-be84-9ced1047ff52-installation-pull-secrets\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:14.798987 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.798924    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbjj6\" (UniqueName: \"kubernetes.io/projected/741d147a-735b-40ef-a55b-8e8f03b062d0-kube-api-access-nbjj6\") pod \"dns-default-v2p2h\" (UID: \"741d147a-735b-40ef-a55b-8e8f03b062d0\") " pod="openshift-dns/dns-default-v2p2h"
Jul 10 22:56:14.799152 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.799123    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-bound-sa-token\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:14.799768 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.799743    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kb2l\" (UniqueName: \"kubernetes.io/projected/67dc0695-fad8-4ecf-bc13-3a7410c1b195-kube-api-access-4kb2l\") pod \"ingress-canary-9ttw7\" (UID: \"67dc0695-fad8-4ecf-bc13-3a7410c1b195\") " pod="openshift-ingress-canary/ingress-canary-9ttw7"
Jul 10 22:56:14.799873 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.799773    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftm6k\" (UniqueName: \"kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-kube-api-access-ftm6k\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:14.876728 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:14.876682    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9ttw7"
Jul 10 22:56:15.292600 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:15.292563    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls\") pod \"dns-default-v2p2h\" (UID: \"741d147a-735b-40ef-a55b-8e8f03b062d0\") " pod="openshift-dns/dns-default-v2p2h"
Jul 10 22:56:15.292756 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:15.292628    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-l9t9h\" (UID: \"2981ee03-5541-484c-8226-2d606103d31e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-l9t9h"
Jul 10 22:56:15.292756 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:15.292703    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:15.292861 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:15.292844    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 22:56:15.292898 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:15.292866    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-769cc77955-d2747: secret "image-registry-tls" not found
Jul 10 22:56:15.292936 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:15.292927    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls podName:e7f0af82-786b-4b48-be84-9ced1047ff52 nodeName:}" failed. No retries permitted until 2026-07-10 22:56:16.292907645 +0000 UTC m=+37.179643786 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls") pod "image-registry-769cc77955-d2747" (UID: "e7f0af82-786b-4b48-be84-9ced1047ff52") : secret "image-registry-tls" not found
Jul 10 22:56:15.293557 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:15.293535    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 22:56:15.293641 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:15.293603    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls podName:741d147a-735b-40ef-a55b-8e8f03b062d0 nodeName:}" failed. No retries permitted until 2026-07-10 22:56:16.293584491 +0000 UTC m=+37.180320631 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls") pod "dns-default-v2p2h" (UID: "741d147a-735b-40ef-a55b-8e8f03b062d0") : secret "dns-default-metrics-tls" not found
Jul 10 22:56:15.293693 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:15.293674    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 22:56:15.293752 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:15.293741    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert podName:2981ee03-5541-484c-8226-2d606103d31e nodeName:}" failed. No retries permitted until 2026-07-10 22:56:16.293724418 +0000 UTC m=+37.180460552 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-l9t9h" (UID: "2981ee03-5541-484c-8226-2d606103d31e") : secret "networking-console-plugin-cert" not found
Jul 10 22:56:15.550426 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:15.550257    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:56:15.550426 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:15.550270    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:56:15.550652 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:15.550594    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:56:15.553116 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:15.553094    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 10 22:56:15.553890 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:15.553118    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 10 22:56:15.554145 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:15.554011    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 10 22:56:15.554145 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:15.554013    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jr855\""
Jul 10 22:56:15.554145 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:15.554055    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9cgdm\""
Jul 10 22:56:15.554145 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:15.554064    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 10 22:56:16.107956 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:16.107777    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9ttw7"]
Jul 10 22:56:16.111230 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:56:16.111201    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67dc0695_fad8_4ecf_bc13_3a7410c1b195.slice/crio-a321d80543b01d4f93b4c115969274a810b119bf772b5d093a989705e9a15c78 WatchSource:0}: Error finding container a321d80543b01d4f93b4c115969274a810b119bf772b5d093a989705e9a15c78: Status 404 returned error can't find the container with id a321d80543b01d4f93b4c115969274a810b119bf772b5d093a989705e9a15c78
Jul 10 22:56:16.302449 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:16.302414    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls\") pod \"dns-default-v2p2h\" (UID: \"741d147a-735b-40ef-a55b-8e8f03b062d0\") " pod="openshift-dns/dns-default-v2p2h"
Jul 10 22:56:16.302646 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:16.302476    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-l9t9h\" (UID: \"2981ee03-5541-484c-8226-2d606103d31e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-l9t9h"
Jul 10 22:56:16.302646 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:16.302548    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:16.302646 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:16.302575    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 22:56:16.302646 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:16.302576    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 22:56:16.302822 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:16.302674    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 22:56:16.302822 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:16.302646    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert podName:2981ee03-5541-484c-8226-2d606103d31e nodeName:}" failed. No retries permitted until 2026-07-10 22:56:18.302629824 +0000 UTC m=+39.189365972 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-l9t9h" (UID: "2981ee03-5541-484c-8226-2d606103d31e") : secret "networking-console-plugin-cert" not found
Jul 10 22:56:16.302822 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:16.302689    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-769cc77955-d2747: secret "image-registry-tls" not found
Jul 10 22:56:16.302822 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:16.302712    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls podName:741d147a-735b-40ef-a55b-8e8f03b062d0 nodeName:}" failed. No retries permitted until 2026-07-10 22:56:18.302693989 +0000 UTC m=+39.189430113 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls") pod "dns-default-v2p2h" (UID: "741d147a-735b-40ef-a55b-8e8f03b062d0") : secret "dns-default-metrics-tls" not found
Jul 10 22:56:16.302822 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:16.302731    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls podName:e7f0af82-786b-4b48-be84-9ced1047ff52 nodeName:}" failed. No retries permitted until 2026-07-10 22:56:18.302718158 +0000 UTC m=+39.189454294 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls") pod "image-registry-769cc77955-d2747" (UID: "e7f0af82-786b-4b48-be84-9ced1047ff52") : secret "image-registry-tls" not found
Jul 10 22:56:16.739267 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:16.739229    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:16.836162 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:16.836117    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:16.889325 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:16.889288    2563 generic.go:356] "Generic (PLEG): container finished" podID="cdbc917d-2667-438e-928b-326933a924d8" containerID="e3456d80921ca165cc279da711203634f098cd8ce0e7ed937cb0b404ff3bc146" exitCode=0
Jul 10 22:56:16.889536 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:16.889403    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xpkzr" event={"ID":"cdbc917d-2667-438e-928b-326933a924d8","Type":"ContainerDied","Data":"e3456d80921ca165cc279da711203634f098cd8ce0e7ed937cb0b404ff3bc146"}
Jul 10 22:56:16.892092 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:16.892064    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9ttw7" event={"ID":"67dc0695-fad8-4ecf-bc13-3a7410c1b195","Type":"ContainerStarted","Data":"a321d80543b01d4f93b4c115969274a810b119bf772b5d093a989705e9a15c78"}
Jul 10 22:56:16.914990 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:16.914952    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:17.898314 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:17.898215    2563 generic.go:356] "Generic (PLEG): container finished" podID="cdbc917d-2667-438e-928b-326933a924d8" containerID="1c76924d56081e7957278e349055e53cb3576525c0964a1713a32329aacb2ab5" exitCode=0
Jul 10 22:56:17.898314 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:17.898260    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xpkzr" event={"ID":"cdbc917d-2667-438e-928b-326933a924d8","Type":"ContainerDied","Data":"1c76924d56081e7957278e349055e53cb3576525c0964a1713a32329aacb2ab5"}
Jul 10 22:56:18.322936 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:18.322891    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls\") pod \"dns-default-v2p2h\" (UID: \"741d147a-735b-40ef-a55b-8e8f03b062d0\") " pod="openshift-dns/dns-default-v2p2h"
Jul 10 22:56:18.323091 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:18.322954    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-l9t9h\" (UID: \"2981ee03-5541-484c-8226-2d606103d31e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-l9t9h"
Jul 10 22:56:18.323091 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:18.323011    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:18.323174 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:18.323122    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 22:56:18.323174 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:18.323133    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-769cc77955-d2747: secret "image-registry-tls" not found
Jul 10 22:56:18.323231 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:18.323177    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls podName:e7f0af82-786b-4b48-be84-9ced1047ff52 nodeName:}" failed. No retries permitted until 2026-07-10 22:56:22.323164083 +0000 UTC m=+43.209900204 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls") pod "image-registry-769cc77955-d2747" (UID: "e7f0af82-786b-4b48-be84-9ced1047ff52") : secret "image-registry-tls" not found
Jul 10 22:56:18.323672 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:18.323638    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 22:56:18.323800 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:18.323695    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls podName:741d147a-735b-40ef-a55b-8e8f03b062d0 nodeName:}" failed. No retries permitted until 2026-07-10 22:56:22.323679502 +0000 UTC m=+43.210415627 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls") pod "dns-default-v2p2h" (UID: "741d147a-735b-40ef-a55b-8e8f03b062d0") : secret "dns-default-metrics-tls" not found
Jul 10 22:56:18.323800 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:18.323758    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 22:56:18.323800 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:18.323793    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert podName:2981ee03-5541-484c-8226-2d606103d31e nodeName:}" failed. No retries permitted until 2026-07-10 22:56:22.323781851 +0000 UTC m=+43.210517975 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-l9t9h" (UID: "2981ee03-5541-484c-8226-2d606103d31e") : secret "networking-console-plugin-cert" not found
Jul 10 22:56:18.803781 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:18.803736    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:18.903977 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:18.903944    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:18.904958 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:18.904937    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xpkzr" event={"ID":"cdbc917d-2667-438e-928b-326933a924d8","Type":"ContainerStarted","Data":"699de5f6d575aa63593e4f2504377b626d30299e97d3838fdea69686110778e7"}
Jul 10 22:56:18.926491 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:18.926432    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xpkzr" podStartSLOduration=8.161403785 podStartE2EDuration="39.926412551s" totalImagesPullingTime="31.765008766s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 22:55:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:56:18.925251369 +0000 UTC m=+39.811987512" watchObservedRunningTime="2026-07-10 22:56:18.926412551 +0000 UTC m=+39.813148693"
Jul 10 22:56:19.017970 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:19.017930    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:19.118599 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:19.118491    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:19.220328 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:19.220294    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:19.909110 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:19.909069    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9ttw7" event={"ID":"67dc0695-fad8-4ecf-bc13-3a7410c1b195","Type":"ContainerStarted","Data":"2925e70213e79b0240213d99310004c4c6fd975a4dd558a850c374959ce30ef2"}
Jul 10 22:56:19.925454 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:19.925397    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-9ttw7" podStartSLOduration=3.174396417 podStartE2EDuration="5.925356138s" totalImagesPullingTime="2.750959721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:56:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:56:19.924582218 +0000 UTC m=+40.811318362" watchObservedRunningTime="2026-07-10 22:56:19.925356138 +0000 UTC m=+40.812092282"
Jul 10 22:56:21.992261 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:21.992211    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:22.094592 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:22.094553    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:22.357979 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:22.357883    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls\") pod \"dns-default-v2p2h\" (UID: \"741d147a-735b-40ef-a55b-8e8f03b062d0\") " pod="openshift-dns/dns-default-v2p2h"
Jul 10 22:56:22.357979 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:22.357936    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-l9t9h\" (UID: \"2981ee03-5541-484c-8226-2d606103d31e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-l9t9h"
Jul 10 22:56:22.358194 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:22.357984    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:22.358194 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:22.358033    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 22:56:22.358194 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:22.358090    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 22:56:22.358194 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:22.358092    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 22:56:22.358194 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:22.358101    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-769cc77955-d2747: secret "image-registry-tls" not found
Jul 10 22:56:22.358194 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:22.358099    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls podName:741d147a-735b-40ef-a55b-8e8f03b062d0 nodeName:}" failed. No retries permitted until 2026-07-10 22:56:30.358082629 +0000 UTC m=+51.244818754 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls") pod "dns-default-v2p2h" (UID: "741d147a-735b-40ef-a55b-8e8f03b062d0") : secret "dns-default-metrics-tls" not found
Jul 10 22:56:22.358194 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:22.358149    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls podName:e7f0af82-786b-4b48-be84-9ced1047ff52 nodeName:}" failed. No retries permitted until 2026-07-10 22:56:30.358137471 +0000 UTC m=+51.244873591 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls") pod "image-registry-769cc77955-d2747" (UID: "e7f0af82-786b-4b48-be84-9ced1047ff52") : secret "image-registry-tls" not found
Jul 10 22:56:22.358194 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:22.358163    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert podName:2981ee03-5541-484c-8226-2d606103d31e nodeName:}" failed. No retries permitted until 2026-07-10 22:56:30.358156605 +0000 UTC m=+51.244892726 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-l9t9h" (UID: "2981ee03-5541-484c-8226-2d606103d31e") : secret "networking-console-plugin-cert" not found
Jul 10 22:56:24.149545 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:24.149461    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:25.361155 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:25.361120    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:25.460831 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:25.460791    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:25.920415 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:25.920379    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:25.969638 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:25.969600    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:26.224353 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:26.224270    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:26.276419 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:26.276385    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:29.540923 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:29.540886    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:29.598016 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:29.597977    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:30.421374 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:30.421323    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:30.421589 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:30.421415    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls\") pod \"dns-default-v2p2h\" (UID: \"741d147a-735b-40ef-a55b-8e8f03b062d0\") " pod="openshift-dns/dns-default-v2p2h"
Jul 10 22:56:30.421589 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:30.421497    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 22:56:30.421589 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:30.421508    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 22:56:30.421589 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:30.421530    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-769cc77955-d2747: secret "image-registry-tls" not found
Jul 10 22:56:30.421589 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:30.421550    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls podName:741d147a-735b-40ef-a55b-8e8f03b062d0 nodeName:}" failed. No retries permitted until 2026-07-10 22:56:46.421536127 +0000 UTC m=+67.308272251 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls") pod "dns-default-v2p2h" (UID: "741d147a-735b-40ef-a55b-8e8f03b062d0") : secret "dns-default-metrics-tls" not found
Jul 10 22:56:30.421589 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:30.421579    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-l9t9h\" (UID: \"2981ee03-5541-484c-8226-2d606103d31e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-l9t9h"
Jul 10 22:56:30.421845 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:30.421593    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls podName:e7f0af82-786b-4b48-be84-9ced1047ff52 nodeName:}" failed. No retries permitted until 2026-07-10 22:56:46.421576334 +0000 UTC m=+67.308312469 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls") pod "image-registry-769cc77955-d2747" (UID: "e7f0af82-786b-4b48-be84-9ced1047ff52") : secret "image-registry-tls" not found
Jul 10 22:56:30.421845 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:30.421658    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 22:56:30.421845 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:30.421707    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert podName:2981ee03-5541-484c-8226-2d606103d31e nodeName:}" failed. No retries permitted until 2026-07-10 22:56:46.421694615 +0000 UTC m=+67.308430740 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-l9t9h" (UID: "2981ee03-5541-484c-8226-2d606103d31e") : secret "networking-console-plugin-cert" not found
Jul 10 22:56:31.864838 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:31.864799    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:31.922554 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:31.922515    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:35.227291 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:35.227250    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:35.284099 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:35.284010    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:40.884496 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:40.884461    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qj86t"
Jul 10 22:56:41.031224 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:41.031190    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:41.086611 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:41.086571    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:44.323406 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:44.323323    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3734910e-9249-4060-afc4-55140bcb9911-original-pull-secret\") pod \"global-pull-secret-syncer-7zb67\" (UID: \"3734910e-9249-4060-afc4-55140bcb9911\") " pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:56:44.323406 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:44.323407    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs\") pod \"network-metrics-daemon-w9kxw\" (UID: \"9829a27f-3dbd-44c7-b278-f26beb69f82a\") " pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:56:44.325943 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:44.325918    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 10 22:56:44.326034 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:44.325920    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 10 22:56:44.333951 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:44.333921    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 10 22:56:44.334019 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:44.334009    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs podName:9829a27f-3dbd-44c7-b278-f26beb69f82a nodeName:}" failed. No retries permitted until 2026-07-10 22:57:48.333991167 +0000 UTC m=+129.220727288 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs") pod "network-metrics-daemon-w9kxw" (UID: "9829a27f-3dbd-44c7-b278-f26beb69f82a") : secret "metrics-daemon-secret" not found
Jul 10 22:56:44.337789 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:44.337748    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3734910e-9249-4060-afc4-55140bcb9911-original-pull-secret\") pod \"global-pull-secret-syncer-7zb67\" (UID: \"3734910e-9249-4060-afc4-55140bcb9911\") " pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:56:44.376417 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:44.376349    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7zb67"
Jul 10 22:56:44.424798 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:44.424753    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8924\" (UniqueName: \"kubernetes.io/projected/3f0e1c5a-a319-437d-9aef-8b3c4150a3b1-kube-api-access-j8924\") pod \"network-check-target-8nt9g\" (UID: \"3f0e1c5a-a319-437d-9aef-8b3c4150a3b1\") " pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:56:44.427487 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:44.427468    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 10 22:56:44.437794 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:44.437765    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 10 22:56:44.448301 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:44.448271    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8924\" (UniqueName: \"kubernetes.io/projected/3f0e1c5a-a319-437d-9aef-8b3c4150a3b1-kube-api-access-j8924\") pod \"network-check-target-8nt9g\" (UID: \"3f0e1c5a-a319-437d-9aef-8b3c4150a3b1\") " pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:56:44.502016 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:44.501976    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7zb67"]
Jul 10 22:56:44.506038 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:56:44.506007    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3734910e_9249_4060_afc4_55140bcb9911.slice/crio-3adccc89901f93f330eb42b92e7875ace2278b955c6e611dcdbce88a47301b10 WatchSource:0}: Error finding container 3adccc89901f93f330eb42b92e7875ace2278b955c6e611dcdbce88a47301b10: Status 404 returned error can't find the container with id 3adccc89901f93f330eb42b92e7875ace2278b955c6e611dcdbce88a47301b10
Jul 10 22:56:44.667103 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:44.667002    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9cgdm\""
Jul 10 22:56:44.673304 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:44.673275    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:56:44.800826 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:44.800794    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8nt9g"]
Jul 10 22:56:44.803842 ip-10-0-141-239 kubenswrapper[2563]: W0710 22:56:44.803805    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f0e1c5a_a319_437d_9aef_8b3c4150a3b1.slice/crio-0b6230a875001f56903497662e743dba3c5cde6674a7620f633d59cbe1e14c32 WatchSource:0}: Error finding container 0b6230a875001f56903497662e743dba3c5cde6674a7620f633d59cbe1e14c32: Status 404 returned error can't find the container with id 0b6230a875001f56903497662e743dba3c5cde6674a7620f633d59cbe1e14c32
Jul 10 22:56:44.968724 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:44.968628    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8nt9g" event={"ID":"3f0e1c5a-a319-437d-9aef-8b3c4150a3b1","Type":"ContainerStarted","Data":"0b6230a875001f56903497662e743dba3c5cde6674a7620f633d59cbe1e14c32"}
Jul 10 22:56:44.970018 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:44.969981    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7zb67" event={"ID":"3734910e-9249-4060-afc4-55140bcb9911","Type":"ContainerStarted","Data":"3adccc89901f93f330eb42b92e7875ace2278b955c6e611dcdbce88a47301b10"}
Jul 10 22:56:46.440607 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:46.440506    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls\") pod \"dns-default-v2p2h\" (UID: \"741d147a-735b-40ef-a55b-8e8f03b062d0\") " pod="openshift-dns/dns-default-v2p2h"
Jul 10 22:56:46.440607 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:46.440572    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-l9t9h\" (UID: \"2981ee03-5541-484c-8226-2d606103d31e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-l9t9h"
Jul 10 22:56:46.441069 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:46.440619    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:56:46.441069 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:46.440689    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 22:56:46.441069 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:46.440728    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 22:56:46.441069 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:46.440730    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 22:56:46.441069 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:46.440787    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-769cc77955-d2747: secret "image-registry-tls" not found
Jul 10 22:56:46.441069 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:46.440798    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls podName:741d147a-735b-40ef-a55b-8e8f03b062d0 nodeName:}" failed. No retries permitted until 2026-07-10 22:57:18.440774449 +0000 UTC m=+99.327510576 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls") pod "dns-default-v2p2h" (UID: "741d147a-735b-40ef-a55b-8e8f03b062d0") : secret "dns-default-metrics-tls" not found
Jul 10 22:56:46.441069 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:46.440825    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls podName:e7f0af82-786b-4b48-be84-9ced1047ff52 nodeName:}" failed. No retries permitted until 2026-07-10 22:57:18.440808226 +0000 UTC m=+99.327544347 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls") pod "image-registry-769cc77955-d2747" (UID: "e7f0af82-786b-4b48-be84-9ced1047ff52") : secret "image-registry-tls" not found
Jul 10 22:56:46.441069 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:56:46.440844    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert podName:2981ee03-5541-484c-8226-2d606103d31e nodeName:}" failed. No retries permitted until 2026-07-10 22:57:18.440835268 +0000 UTC m=+99.327571395 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-l9t9h" (UID: "2981ee03-5541-484c-8226-2d606103d31e") : secret "networking-console-plugin-cert" not found
Jul 10 22:56:47.352987 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:47.352940    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:47.402820 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:47.402782    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:49.985122 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:49.985078    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8nt9g" event={"ID":"3f0e1c5a-a319-437d-9aef-8b3c4150a3b1","Type":"ContainerStarted","Data":"60327a445c557836af0f54e08476734f70afe80af256401eabba80201cf2d742"}
Jul 10 22:56:49.985617 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:49.985188    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:56:49.986731 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:49.986699    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7zb67" event={"ID":"3734910e-9249-4060-afc4-55140bcb9911","Type":"ContainerStarted","Data":"ad899db39e4e955393b59c2dfbe8b037102e04a36b15b64dea36a19ce01457fd"}
Jul 10 22:56:49.999546 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:49.999493    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8nt9g" podStartSLOduration=66.148033563 podStartE2EDuration="1m10.999476105s" totalImagesPullingTime="4.851442542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:56:49.998756465 +0000 UTC m=+70.885492609" watchObservedRunningTime="2026-07-10 22:56:49.999476105 +0000 UTC m=+70.886212271"
Jul 10 22:56:50.012596 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:50.012536    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7zb67" podStartSLOduration=65.852662611 podStartE2EDuration="1m11.012517612s" totalImagesPullingTime="5.159855001s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 22:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 22:56:50.011660219 +0000 UTC m=+70.898396363" watchObservedRunningTime="2026-07-10 22:56:50.012517612 +0000 UTC m=+70.899253755"
Jul 10 22:56:54.816403 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:54.816371    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:56:54.918546 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:56:54.918508    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:57:18.483176 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:57:18.483137    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls\") pod \"dns-default-v2p2h\" (UID: \"741d147a-735b-40ef-a55b-8e8f03b062d0\") " pod="openshift-dns/dns-default-v2p2h"
Jul 10 22:57:18.483176 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:57:18.483192    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-l9t9h\" (UID: \"2981ee03-5541-484c-8226-2d606103d31e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-l9t9h"
Jul 10 22:57:18.483791 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:57:18.483287    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:57:18.483791 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:57:18.483312    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 22:57:18.483791 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:57:18.483412    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert podName:2981ee03-5541-484c-8226-2d606103d31e nodeName:}" failed. No retries permitted until 2026-07-10 22:58:22.483390081 +0000 UTC m=+163.370126203 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-l9t9h" (UID: "2981ee03-5541-484c-8226-2d606103d31e") : secret "networking-console-plugin-cert" not found
Jul 10 22:57:18.483791 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:57:18.483423    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 22:57:18.483791 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:57:18.483438    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-769cc77955-d2747: secret "image-registry-tls" not found
Jul 10 22:57:18.483791 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:57:18.483497    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls podName:e7f0af82-786b-4b48-be84-9ced1047ff52 nodeName:}" failed. No retries permitted until 2026-07-10 22:58:22.48348105 +0000 UTC m=+163.370217191 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls") pod "image-registry-769cc77955-d2747" (UID: "e7f0af82-786b-4b48-be84-9ced1047ff52") : secret "image-registry-tls" not found
Jul 10 22:57:18.483791 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:57:18.483502    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 22:57:18.483791 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:57:18.483545    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls podName:741d147a-735b-40ef-a55b-8e8f03b062d0 nodeName:}" failed. No retries permitted until 2026-07-10 22:58:22.48353497 +0000 UTC m=+163.370271092 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls") pod "dns-default-v2p2h" (UID: "741d147a-735b-40ef-a55b-8e8f03b062d0") : secret "dns-default-metrics-tls" not found
Jul 10 22:57:20.991842 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:57:20.991809    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8nt9g"
Jul 10 22:57:48.403470 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:57:48.403421    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs\") pod \"network-metrics-daemon-w9kxw\" (UID: \"9829a27f-3dbd-44c7-b278-f26beb69f82a\") " pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:57:48.404046 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:57:48.403631    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 10 22:57:48.404046 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:57:48.403734    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs podName:9829a27f-3dbd-44c7-b278-f26beb69f82a nodeName:}" failed. No retries permitted until 2026-07-10 22:59:50.403709728 +0000 UTC m=+251.290445852 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs") pod "network-metrics-daemon-w9kxw" (UID: "9829a27f-3dbd-44c7-b278-f26beb69f82a") : secret "metrics-daemon-secret" not found
Jul 10 22:58:17.555893 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:58:17.555848    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cf856758c-l9t9h" podUID="2981ee03-5541-484c-8226-2d606103d31e"
Jul 10 22:58:17.564983 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:58:17.564947    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-769cc77955-d2747" podUID="e7f0af82-786b-4b48-be84-9ced1047ff52"
Jul 10 22:58:17.581177 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:58:17.581137    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-v2p2h" podUID="741d147a-735b-40ef-a55b-8e8f03b062d0"
Jul 10 22:58:18.198550 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:58:18.198516    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:58:18.198714 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:58:18.198516    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v2p2h"
Jul 10 22:58:18.198714 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:58:18.198526    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-l9t9h"
Jul 10 22:58:18.570586 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:58:18.570546    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-w9kxw" podUID="9829a27f-3dbd-44c7-b278-f26beb69f82a"
Jul 10 22:58:22.560374 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:58:22.560326    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-l9t9h\" (UID: \"2981ee03-5541-484c-8226-2d606103d31e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-l9t9h"
Jul 10 22:58:22.560877 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:58:22.560432    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 22:58:22.560877 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:58:22.560487    2563 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 22:58:22.560877 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:58:22.560494    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls\") pod \"dns-default-v2p2h\" (UID: \"741d147a-735b-40ef-a55b-8e8f03b062d0\") " pod="openshift-dns/dns-default-v2p2h"
Jul 10 22:58:22.560877 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:58:22.560581    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 22:58:22.560877 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:58:22.560596    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 22:58:22.560877 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:58:22.560585    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert podName:2981ee03-5541-484c-8226-2d606103d31e nodeName:}" failed. No retries permitted until 2026-07-10 23:00:24.560565133 +0000 UTC m=+285.447301279 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-l9t9h" (UID: "2981ee03-5541-484c-8226-2d606103d31e") : secret "networking-console-plugin-cert" not found
Jul 10 22:58:22.560877 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:58:22.560619    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-769cc77955-d2747: secret "image-registry-tls" not found
Jul 10 22:58:22.560877 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:58:22.560631    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls podName:741d147a-735b-40ef-a55b-8e8f03b062d0 nodeName:}" failed. No retries permitted until 2026-07-10 23:00:24.560620933 +0000 UTC m=+285.447357055 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls") pod "dns-default-v2p2h" (UID: "741d147a-735b-40ef-a55b-8e8f03b062d0") : secret "dns-default-metrics-tls" not found
Jul 10 22:58:22.560877 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:58:22.560700    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls podName:e7f0af82-786b-4b48-be84-9ced1047ff52 nodeName:}" failed. No retries permitted until 2026-07-10 23:00:24.56068098 +0000 UTC m=+285.447417112 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls") pod "image-registry-769cc77955-d2747" (UID: "e7f0af82-786b-4b48-be84-9ced1047ff52") : secret "image-registry-tls" not found
Jul 10 22:58:32.550494 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:58:32.550408    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:58:35.112059 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:58:35.112030    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:59:18.719911 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:59:18.719875    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 22:59:50.497850 ip-10-0-141-239 kubenswrapper[2563]: I0710 22:59:50.497807    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs\") pod \"network-metrics-daemon-w9kxw\" (UID: \"9829a27f-3dbd-44c7-b278-f26beb69f82a\") " pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 22:59:50.498344 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:59:50.497941    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 10 22:59:50.498344 ip-10-0-141-239 kubenswrapper[2563]: E0710 22:59:50.497995    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs podName:9829a27f-3dbd-44c7-b278-f26beb69f82a nodeName:}" failed. No retries permitted until 2026-07-10 23:01:52.497980944 +0000 UTC m=+373.384717065 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs") pod "network-metrics-daemon-w9kxw" (UID: "9829a27f-3dbd-44c7-b278-f26beb69f82a") : secret "metrics-daemon-secret" not found
Jul 10 23:00:07.082504 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.082420    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tf99c"]
Jul 10 23:00:07.085420 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.085389    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tf99c"
Jul 10 23:00:07.086014 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.085991    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-24jvw"]
Jul 10 23:00:07.087709 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.087693    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 10 23:00:07.088572 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.088557    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 10 23:00:07.088717 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.088575    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-x4zqj\""
Jul 10 23:00:07.088717 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.088580    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 10 23:00:07.088717 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.088616    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 10 23:00:07.088885 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.088773    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-24jvw"
Jul 10 23:00:07.091353 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.091309    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-v8tl6\""
Jul 10 23:00:07.091546 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.091510    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 10 23:00:07.092141 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.092117    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 10 23:00:07.092858 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.092796    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bxm84"]
Jul 10 23:00:07.095542 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.095517    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tf99c"]
Jul 10 23:00:07.095653 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.095622    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bxm84"
Jul 10 23:00:07.098095 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.098072    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 10 23:00:07.098469 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.098451    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 10 23:00:07.098592 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.098457    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-lgvm9\""
Jul 10 23:00:07.099008 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.098654    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 10 23:00:07.099105 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.099059    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 10 23:00:07.099252 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.099230    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-24jvw"]
Jul 10 23:00:07.107856 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.107830    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bxm84"]
Jul 10 23:00:07.179605 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.179568    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-dg56q"]
Jul 10 23:00:07.182684 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.182657    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-dg56q"
Jul 10 23:00:07.183505 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.183484    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-hwd7d"]
Jul 10 23:00:07.185002 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.184982    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-77vd6\""
Jul 10 23:00:07.186160 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.186145    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9tbg6"]
Jul 10 23:00:07.186286 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.186269    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-hwd7d"
Jul 10 23:00:07.188896 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.188863    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9tbg6"
Jul 10 23:00:07.189011 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.188899    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 10 23:00:07.189011 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.188942    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 10 23:00:07.189011 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.188977    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 10 23:00:07.189011 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.189003    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 10 23:00:07.189203 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.189094    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-92d7f\""
Jul 10 23:00:07.191004 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.190987    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-4lg9r\""
Jul 10 23:00:07.191661 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.191645    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 10 23:00:07.192394 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.192347    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 10 23:00:07.192489 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.192414    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 10 23:00:07.193053 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.193036    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 10 23:00:07.194557 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.194535    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 10 23:00:07.196393 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.196306    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-dg56q"]
Jul 10 23:00:07.197541 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.197521    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-hwd7d"]
Jul 10 23:00:07.199016 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.198842    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9tbg6"]
Jul 10 23:00:07.222576 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.222538    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5dd8a036-d481-4d56-9e10-1dc8e52983ac-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-bxm84\" (UID: \"5dd8a036-d481-4d56-9e10-1dc8e52983ac\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bxm84"
Jul 10 23:00:07.222576 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.222583    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/db2cc68f-5886-4ce1-8807-cee1d5760f99-config\") pod \"service-ca-operator-7695fc6c8c-tf99c\" (UID: \"db2cc68f-5886-4ce1-8807-cee1d5760f99\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tf99c"
Jul 10 23:00:07.222800 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.222604    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twfl8\" (UniqueName: \"kubernetes.io/projected/5dd8a036-d481-4d56-9e10-1dc8e52983ac-kube-api-access-twfl8\") pod \"kube-storage-version-migrator-operator-799cb7f958-bxm84\" (UID: \"5dd8a036-d481-4d56-9e10-1dc8e52983ac\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bxm84"
Jul 10 23:00:07.222800 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.222693    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5q6c\" (UniqueName: \"kubernetes.io/projected/76f2a250-6c88-458c-91dc-b89e97416adc-kube-api-access-c5q6c\") pod \"volume-data-source-validator-78495597f6-24jvw\" (UID: \"76f2a250-6c88-458c-91dc-b89e97416adc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-24jvw"
Jul 10 23:00:07.222800 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.222718    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5dd8a036-d481-4d56-9e10-1dc8e52983ac-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-bxm84\" (UID: \"5dd8a036-d481-4d56-9e10-1dc8e52983ac\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bxm84"
Jul 10 23:00:07.222800 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.222741    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pj47\" (UniqueName: \"kubernetes.io/projected/db2cc68f-5886-4ce1-8807-cee1d5760f99-kube-api-access-9pj47\") pod \"service-ca-operator-7695fc6c8c-tf99c\" (UID: \"db2cc68f-5886-4ce1-8807-cee1d5760f99\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tf99c"
Jul 10 23:00:07.222800 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.222769    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/db2cc68f-5886-4ce1-8807-cee1d5760f99-serving-cert\") pod \"service-ca-operator-7695fc6c8c-tf99c\" (UID: \"db2cc68f-5886-4ce1-8807-cee1d5760f99\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tf99c"
Jul 10 23:00:07.323249 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.323204    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/24725b9a-723c-41d5-959d-4ae184f4d456-serving-cert\") pod \"insights-operator-68d9dd94cf-hwd7d\" (UID: \"24725b9a-723c-41d5-959d-4ae184f4d456\") " pod="openshift-insights/insights-operator-68d9dd94cf-hwd7d"
Jul 10 23:00:07.323249 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.323254    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5dd8a036-d481-4d56-9e10-1dc8e52983ac-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-bxm84\" (UID: \"5dd8a036-d481-4d56-9e10-1dc8e52983ac\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bxm84"
Jul 10 23:00:07.323556 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.323335    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmmj8\" (UniqueName: \"kubernetes.io/projected/dae10db7-61ca-4e1e-80f2-ab4fb81ebde2-kube-api-access-xmmj8\") pod \"cluster-monitoring-operator-5f7f494dd4-9tbg6\" (UID: \"dae10db7-61ca-4e1e-80f2-ab4fb81ebde2\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9tbg6"
Jul 10 23:00:07.323556 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.323394    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5dd8a036-d481-4d56-9e10-1dc8e52983ac-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-bxm84\" (UID: \"5dd8a036-d481-4d56-9e10-1dc8e52983ac\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bxm84"
Jul 10 23:00:07.323556 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.323421    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/dae10db7-61ca-4e1e-80f2-ab4fb81ebde2-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-9tbg6\" (UID: \"dae10db7-61ca-4e1e-80f2-ab4fb81ebde2\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9tbg6"
Jul 10 23:00:07.323556 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.323444    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/24725b9a-723c-41d5-959d-4ae184f4d456-snapshots\") pod \"insights-operator-68d9dd94cf-hwd7d\" (UID: \"24725b9a-723c-41d5-959d-4ae184f4d456\") " pod="openshift-insights/insights-operator-68d9dd94cf-hwd7d"
Jul 10 23:00:07.323556 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.323466    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24725b9a-723c-41d5-959d-4ae184f4d456-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-hwd7d\" (UID: \"24725b9a-723c-41d5-959d-4ae184f4d456\") " pod="openshift-insights/insights-operator-68d9dd94cf-hwd7d"
Jul 10 23:00:07.323556 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.323530    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24725b9a-723c-41d5-959d-4ae184f4d456-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-hwd7d\" (UID: \"24725b9a-723c-41d5-959d-4ae184f4d456\") " pod="openshift-insights/insights-operator-68d9dd94cf-hwd7d"
Jul 10 23:00:07.323818 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.323602    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sthn2\" (UniqueName: \"kubernetes.io/projected/24725b9a-723c-41d5-959d-4ae184f4d456-kube-api-access-sthn2\") pod \"insights-operator-68d9dd94cf-hwd7d\" (UID: \"24725b9a-723c-41d5-959d-4ae184f4d456\") " pod="openshift-insights/insights-operator-68d9dd94cf-hwd7d"
Jul 10 23:00:07.323818 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.323628    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dae10db7-61ca-4e1e-80f2-ab4fb81ebde2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-9tbg6\" (UID: \"dae10db7-61ca-4e1e-80f2-ab4fb81ebde2\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9tbg6"
Jul 10 23:00:07.323818 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.323651    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twfl8\" (UniqueName: \"kubernetes.io/projected/5dd8a036-d481-4d56-9e10-1dc8e52983ac-kube-api-access-twfl8\") pod \"kube-storage-version-migrator-operator-799cb7f958-bxm84\" (UID: \"5dd8a036-d481-4d56-9e10-1dc8e52983ac\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bxm84"
Jul 10 23:00:07.323818 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.323713    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk962\" (UniqueName: \"kubernetes.io/projected/3c6ae14a-0796-4ee6-938b-0f573c34fa29-kube-api-access-pk962\") pod \"network-check-source-869dcfcdcd-dg56q\" (UID: \"3c6ae14a-0796-4ee6-938b-0f573c34fa29\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-dg56q"
Jul 10 23:00:07.323818 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.323740    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/db2cc68f-5886-4ce1-8807-cee1d5760f99-serving-cert\") pod \"service-ca-operator-7695fc6c8c-tf99c\" (UID: \"db2cc68f-5886-4ce1-8807-cee1d5760f99\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tf99c"
Jul 10 23:00:07.323818 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.323780    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/db2cc68f-5886-4ce1-8807-cee1d5760f99-config\") pod \"service-ca-operator-7695fc6c8c-tf99c\" (UID: \"db2cc68f-5886-4ce1-8807-cee1d5760f99\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tf99c"
Jul 10 23:00:07.323818 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.323803    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pj47\" (UniqueName: \"kubernetes.io/projected/db2cc68f-5886-4ce1-8807-cee1d5760f99-kube-api-access-9pj47\") pod \"service-ca-operator-7695fc6c8c-tf99c\" (UID: \"db2cc68f-5886-4ce1-8807-cee1d5760f99\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tf99c"
Jul 10 23:00:07.324174 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.323829    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24725b9a-723c-41d5-959d-4ae184f4d456-tmp\") pod \"insights-operator-68d9dd94cf-hwd7d\" (UID: \"24725b9a-723c-41d5-959d-4ae184f4d456\") " pod="openshift-insights/insights-operator-68d9dd94cf-hwd7d"
Jul 10 23:00:07.324174 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.323857    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5q6c\" (UniqueName: \"kubernetes.io/projected/76f2a250-6c88-458c-91dc-b89e97416adc-kube-api-access-c5q6c\") pod \"volume-data-source-validator-78495597f6-24jvw\" (UID: \"76f2a250-6c88-458c-91dc-b89e97416adc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-24jvw"
Jul 10 23:00:07.324174 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.323889    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5dd8a036-d481-4d56-9e10-1dc8e52983ac-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-bxm84\" (UID: \"5dd8a036-d481-4d56-9e10-1dc8e52983ac\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bxm84"
Jul 10 23:00:07.324555 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.324529    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/db2cc68f-5886-4ce1-8807-cee1d5760f99-config\") pod \"service-ca-operator-7695fc6c8c-tf99c\" (UID: \"db2cc68f-5886-4ce1-8807-cee1d5760f99\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tf99c"
Jul 10 23:00:07.324642 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.324602    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:07.325846 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.325821    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5dd8a036-d481-4d56-9e10-1dc8e52983ac-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-bxm84\" (UID: \"5dd8a036-d481-4d56-9e10-1dc8e52983ac\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bxm84"
Jul 10 23:00:07.326482 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.326465    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/db2cc68f-5886-4ce1-8807-cee1d5760f99-serving-cert\") pod \"service-ca-operator-7695fc6c8c-tf99c\" (UID: \"db2cc68f-5886-4ce1-8807-cee1d5760f99\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tf99c"
Jul 10 23:00:07.332138 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.332101    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twfl8\" (UniqueName: \"kubernetes.io/projected/5dd8a036-d481-4d56-9e10-1dc8e52983ac-kube-api-access-twfl8\") pod \"kube-storage-version-migrator-operator-799cb7f958-bxm84\" (UID: \"5dd8a036-d481-4d56-9e10-1dc8e52983ac\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bxm84"
Jul 10 23:00:07.332493 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.332475    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pj47\" (UniqueName: \"kubernetes.io/projected/db2cc68f-5886-4ce1-8807-cee1d5760f99-kube-api-access-9pj47\") pod \"service-ca-operator-7695fc6c8c-tf99c\" (UID: \"db2cc68f-5886-4ce1-8807-cee1d5760f99\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tf99c"
Jul 10 23:00:07.332616 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.332579    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5q6c\" (UniqueName: \"kubernetes.io/projected/76f2a250-6c88-458c-91dc-b89e97416adc-kube-api-access-c5q6c\") pod \"volume-data-source-validator-78495597f6-24jvw\" (UID: \"76f2a250-6c88-458c-91dc-b89e97416adc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-24jvw"
Jul 10 23:00:07.397099 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.397052    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tf99c"
Jul 10 23:00:07.404275 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.404240    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-24jvw"
Jul 10 23:00:07.409423 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.409396    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bxm84"
Jul 10 23:00:07.424270 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.424234    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/24725b9a-723c-41d5-959d-4ae184f4d456-serving-cert\") pod \"insights-operator-68d9dd94cf-hwd7d\" (UID: \"24725b9a-723c-41d5-959d-4ae184f4d456\") " pod="openshift-insights/insights-operator-68d9dd94cf-hwd7d"
Jul 10 23:00:07.424470 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.424322    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmmj8\" (UniqueName: \"kubernetes.io/projected/dae10db7-61ca-4e1e-80f2-ab4fb81ebde2-kube-api-access-xmmj8\") pod \"cluster-monitoring-operator-5f7f494dd4-9tbg6\" (UID: \"dae10db7-61ca-4e1e-80f2-ab4fb81ebde2\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9tbg6"
Jul 10 23:00:07.424470 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.424387    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/dae10db7-61ca-4e1e-80f2-ab4fb81ebde2-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-9tbg6\" (UID: \"dae10db7-61ca-4e1e-80f2-ab4fb81ebde2\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9tbg6"
Jul 10 23:00:07.424470 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.424412    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/24725b9a-723c-41d5-959d-4ae184f4d456-snapshots\") pod \"insights-operator-68d9dd94cf-hwd7d\" (UID: \"24725b9a-723c-41d5-959d-4ae184f4d456\") " pod="openshift-insights/insights-operator-68d9dd94cf-hwd7d"
Jul 10 23:00:07.424470 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.424437    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24725b9a-723c-41d5-959d-4ae184f4d456-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-hwd7d\" (UID: \"24725b9a-723c-41d5-959d-4ae184f4d456\") " pod="openshift-insights/insights-operator-68d9dd94cf-hwd7d"
Jul 10 23:00:07.424674 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.424478    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24725b9a-723c-41d5-959d-4ae184f4d456-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-hwd7d\" (UID: \"24725b9a-723c-41d5-959d-4ae184f4d456\") " pod="openshift-insights/insights-operator-68d9dd94cf-hwd7d"
Jul 10 23:00:07.424674 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.424508    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sthn2\" (UniqueName: \"kubernetes.io/projected/24725b9a-723c-41d5-959d-4ae184f4d456-kube-api-access-sthn2\") pod \"insights-operator-68d9dd94cf-hwd7d\" (UID: \"24725b9a-723c-41d5-959d-4ae184f4d456\") " pod="openshift-insights/insights-operator-68d9dd94cf-hwd7d"
Jul 10 23:00:07.424674 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.424531    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dae10db7-61ca-4e1e-80f2-ab4fb81ebde2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-9tbg6\" (UID: \"dae10db7-61ca-4e1e-80f2-ab4fb81ebde2\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9tbg6"
Jul 10 23:00:07.424674 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.424582    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk962\" (UniqueName: \"kubernetes.io/projected/3c6ae14a-0796-4ee6-938b-0f573c34fa29-kube-api-access-pk962\") pod \"network-check-source-869dcfcdcd-dg56q\" (UID: \"3c6ae14a-0796-4ee6-938b-0f573c34fa29\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-dg56q"
Jul 10 23:00:07.424674 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.424625    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24725b9a-723c-41d5-959d-4ae184f4d456-tmp\") pod \"insights-operator-68d9dd94cf-hwd7d\" (UID: \"24725b9a-723c-41d5-959d-4ae184f4d456\") " pod="openshift-insights/insights-operator-68d9dd94cf-hwd7d"
Jul 10 23:00:07.425673 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.425023    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24725b9a-723c-41d5-959d-4ae184f4d456-tmp\") pod \"insights-operator-68d9dd94cf-hwd7d\" (UID: \"24725b9a-723c-41d5-959d-4ae184f4d456\") " pod="openshift-insights/insights-operator-68d9dd94cf-hwd7d"
Jul 10 23:00:07.425673 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.425148    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:07.425673 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:07.425308    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 10 23:00:07.425673 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:07.425398    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dae10db7-61ca-4e1e-80f2-ab4fb81ebde2-cluster-monitoring-operator-tls podName:dae10db7-61ca-4e1e-80f2-ab4fb81ebde2 nodeName:}" failed. No retries permitted until 2026-07-10 23:00:07.925379077 +0000 UTC m=+268.812115212 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/dae10db7-61ca-4e1e-80f2-ab4fb81ebde2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-9tbg6" (UID: "dae10db7-61ca-4e1e-80f2-ab4fb81ebde2") : secret "cluster-monitoring-operator-tls" not found
Jul 10 23:00:07.425673 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.425513    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24725b9a-723c-41d5-959d-4ae184f4d456-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-hwd7d\" (UID: \"24725b9a-723c-41d5-959d-4ae184f4d456\") " pod="openshift-insights/insights-operator-68d9dd94cf-hwd7d"
Jul 10 23:00:07.425673 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.425591    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/24725b9a-723c-41d5-959d-4ae184f4d456-snapshots\") pod \"insights-operator-68d9dd94cf-hwd7d\" (UID: \"24725b9a-723c-41d5-959d-4ae184f4d456\") " pod="openshift-insights/insights-operator-68d9dd94cf-hwd7d"
Jul 10 23:00:07.426057 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.425745    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/dae10db7-61ca-4e1e-80f2-ab4fb81ebde2-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-9tbg6\" (UID: \"dae10db7-61ca-4e1e-80f2-ab4fb81ebde2\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9tbg6"
Jul 10 23:00:07.426382 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.426341    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24725b9a-723c-41d5-959d-4ae184f4d456-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-hwd7d\" (UID: \"24725b9a-723c-41d5-959d-4ae184f4d456\") " pod="openshift-insights/insights-operator-68d9dd94cf-hwd7d"
Jul 10 23:00:07.428048 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.428014    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/24725b9a-723c-41d5-959d-4ae184f4d456-serving-cert\") pod \"insights-operator-68d9dd94cf-hwd7d\" (UID: \"24725b9a-723c-41d5-959d-4ae184f4d456\") " pod="openshift-insights/insights-operator-68d9dd94cf-hwd7d"
Jul 10 23:00:07.432506 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.432444    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmmj8\" (UniqueName: \"kubernetes.io/projected/dae10db7-61ca-4e1e-80f2-ab4fb81ebde2-kube-api-access-xmmj8\") pod \"cluster-monitoring-operator-5f7f494dd4-9tbg6\" (UID: \"dae10db7-61ca-4e1e-80f2-ab4fb81ebde2\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9tbg6"
Jul 10 23:00:07.433971 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.433943    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sthn2\" (UniqueName: \"kubernetes.io/projected/24725b9a-723c-41d5-959d-4ae184f4d456-kube-api-access-sthn2\") pod \"insights-operator-68d9dd94cf-hwd7d\" (UID: \"24725b9a-723c-41d5-959d-4ae184f4d456\") " pod="openshift-insights/insights-operator-68d9dd94cf-hwd7d"
Jul 10 23:00:07.434429 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.434408    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk962\" (UniqueName: \"kubernetes.io/projected/3c6ae14a-0796-4ee6-938b-0f573c34fa29-kube-api-access-pk962\") pod \"network-check-source-869dcfcdcd-dg56q\" (UID: \"3c6ae14a-0796-4ee6-938b-0f573c34fa29\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-dg56q"
Jul 10 23:00:07.493696 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.493587    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-dg56q"
Jul 10 23:00:07.503067 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.503027    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-hwd7d"
Jul 10 23:00:07.560062 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.560032    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tf99c"]
Jul 10 23:00:07.565648 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:00:07.565101    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb2cc68f_5886_4ce1_8807_cee1d5760f99.slice/crio-27cbb0c14b342ae30cd5dc07e8a0fc029de376b2fbd1f14e3e67f4ffad2603fc WatchSource:0}: Error finding container 27cbb0c14b342ae30cd5dc07e8a0fc029de376b2fbd1f14e3e67f4ffad2603fc: Status 404 returned error can't find the container with id 27cbb0c14b342ae30cd5dc07e8a0fc029de376b2fbd1f14e3e67f4ffad2603fc
Jul 10 23:00:07.583026 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.582944    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bxm84"]
Jul 10 23:00:07.586480 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.586447    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-24jvw"]
Jul 10 23:00:07.654305 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.654273    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-dg56q"]
Jul 10 23:00:07.657668 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:00:07.657636    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c6ae14a_0796_4ee6_938b_0f573c34fa29.slice/crio-eb1d9c9d199c69e8921044b1fbb244d82857bbc3294f6267cab97826d384ef1b WatchSource:0}: Error finding container eb1d9c9d199c69e8921044b1fbb244d82857bbc3294f6267cab97826d384ef1b: Status 404 returned error can't find the container with id eb1d9c9d199c69e8921044b1fbb244d82857bbc3294f6267cab97826d384ef1b
Jul 10 23:00:07.669938 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.669905    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-hwd7d"]
Jul 10 23:00:07.673001 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:00:07.672921    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24725b9a_723c_41d5_959d_4ae184f4d456.slice/crio-7c2e725d10a7815bb09e48afdd6d0c0c90167e7fa4816b6989eedd60c7042dbe WatchSource:0}: Error finding container 7c2e725d10a7815bb09e48afdd6d0c0c90167e7fa4816b6989eedd60c7042dbe: Status 404 returned error can't find the container with id 7c2e725d10a7815bb09e48afdd6d0c0c90167e7fa4816b6989eedd60c7042dbe
Jul 10 23:00:07.928035 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:07.927928    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dae10db7-61ca-4e1e-80f2-ab4fb81ebde2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-9tbg6\" (UID: \"dae10db7-61ca-4e1e-80f2-ab4fb81ebde2\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9tbg6"
Jul 10 23:00:07.928188 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:07.928087    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 10 23:00:07.928188 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:07.928158    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dae10db7-61ca-4e1e-80f2-ab4fb81ebde2-cluster-monitoring-operator-tls podName:dae10db7-61ca-4e1e-80f2-ab4fb81ebde2 nodeName:}" failed. No retries permitted until 2026-07-10 23:00:08.928140859 +0000 UTC m=+269.814876980 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/dae10db7-61ca-4e1e-80f2-ab4fb81ebde2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-9tbg6" (UID: "dae10db7-61ca-4e1e-80f2-ab4fb81ebde2") : secret "cluster-monitoring-operator-tls" not found
Jul 10 23:00:08.456028 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:08.455987    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-hwd7d" event={"ID":"24725b9a-723c-41d5-959d-4ae184f4d456","Type":"ContainerStarted","Data":"7c2e725d10a7815bb09e48afdd6d0c0c90167e7fa4816b6989eedd60c7042dbe"}
Jul 10 23:00:08.458124 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:08.457991    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bxm84" event={"ID":"5dd8a036-d481-4d56-9e10-1dc8e52983ac","Type":"ContainerStarted","Data":"d4713516b09637033acc9e8c2cd398b21d0aab9e927ab77af16f9134b69334a1"}
Jul 10 23:00:08.460878 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:08.460807    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tf99c" event={"ID":"db2cc68f-5886-4ce1-8807-cee1d5760f99","Type":"ContainerStarted","Data":"27cbb0c14b342ae30cd5dc07e8a0fc029de376b2fbd1f14e3e67f4ffad2603fc"}
Jul 10 23:00:08.464467 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:08.463580    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-dg56q" event={"ID":"3c6ae14a-0796-4ee6-938b-0f573c34fa29","Type":"ContainerStarted","Data":"d6ff75195b7b0b12fee5d5a6c6ef69bb67f67eb10657b680e634ad1339acd1e6"}
Jul 10 23:00:08.464467 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:08.463642    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-dg56q" event={"ID":"3c6ae14a-0796-4ee6-938b-0f573c34fa29","Type":"ContainerStarted","Data":"eb1d9c9d199c69e8921044b1fbb244d82857bbc3294f6267cab97826d384ef1b"}
Jul 10 23:00:08.466734 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:08.466701    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-24jvw" event={"ID":"76f2a250-6c88-458c-91dc-b89e97416adc","Type":"ContainerStarted","Data":"d7322daa7640544d44741a09af5df384cd38ea4383d53901a4bd61c1dda6bc15"}
Jul 10 23:00:08.479625 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:08.479166    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-dg56q" podStartSLOduration=1.479141291 podStartE2EDuration="1.479141291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:00:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:00:08.477566918 +0000 UTC m=+269.364303062" watchObservedRunningTime="2026-07-10 23:00:08.479141291 +0000 UTC m=+269.365877434"
Jul 10 23:00:08.944148 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:08.943434    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dae10db7-61ca-4e1e-80f2-ab4fb81ebde2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-9tbg6\" (UID: \"dae10db7-61ca-4e1e-80f2-ab4fb81ebde2\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9tbg6"
Jul 10 23:00:08.944148 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:08.943723    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 10 23:00:08.944148 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:08.943792    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dae10db7-61ca-4e1e-80f2-ab4fb81ebde2-cluster-monitoring-operator-tls podName:dae10db7-61ca-4e1e-80f2-ab4fb81ebde2 nodeName:}" failed. No retries permitted until 2026-07-10 23:00:10.943772686 +0000 UTC m=+271.830508813 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/dae10db7-61ca-4e1e-80f2-ab4fb81ebde2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-9tbg6" (UID: "dae10db7-61ca-4e1e-80f2-ab4fb81ebde2") : secret "cluster-monitoring-operator-tls" not found
Jul 10 23:00:10.959729 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:10.959683    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dae10db7-61ca-4e1e-80f2-ab4fb81ebde2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-9tbg6\" (UID: \"dae10db7-61ca-4e1e-80f2-ab4fb81ebde2\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9tbg6"
Jul 10 23:00:10.960105 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:10.959840    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 10 23:00:10.960105 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:10.959905    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dae10db7-61ca-4e1e-80f2-ab4fb81ebde2-cluster-monitoring-operator-tls podName:dae10db7-61ca-4e1e-80f2-ab4fb81ebde2 nodeName:}" failed. No retries permitted until 2026-07-10 23:00:14.959891263 +0000 UTC m=+275.846627383 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/dae10db7-61ca-4e1e-80f2-ab4fb81ebde2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-9tbg6" (UID: "dae10db7-61ca-4e1e-80f2-ab4fb81ebde2") : secret "cluster-monitoring-operator-tls" not found
Jul 10 23:00:11.478067 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:11.478024    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-hwd7d" event={"ID":"24725b9a-723c-41d5-959d-4ae184f4d456","Type":"ContainerStarted","Data":"d500df93f620bbded7576f62dadd04dedaf4d6101d25ff7c73ff9e798989cade"}
Jul 10 23:00:11.479784 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:11.479755    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bxm84" event={"ID":"5dd8a036-d481-4d56-9e10-1dc8e52983ac","Type":"ContainerStarted","Data":"92b33daf1228843c502f0e16df0a84e1b0312063fd34eb2f2de853646a83d10b"}
Jul 10 23:00:11.481530 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:11.481467    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tf99c" event={"ID":"db2cc68f-5886-4ce1-8807-cee1d5760f99","Type":"ContainerStarted","Data":"067a384822b61f2291e8c802ae61ce68d492e7462eb89e14906d47a9cf8601bb"}
Jul 10 23:00:11.483081 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:11.483058    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-24jvw" event={"ID":"76f2a250-6c88-458c-91dc-b89e97416adc","Type":"ContainerStarted","Data":"53cfccfc0297b17e77c25d9f6ea10634c21818692e99011369fdbacc4ee56597"}
Jul 10 23:00:11.493480 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:11.493428    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-hwd7d" podStartSLOduration=1.376945417 podStartE2EDuration="4.493407427s" totalImagesPullingTime="3.11646201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:00:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:00:11.492970409 +0000 UTC m=+272.379706553" watchObservedRunningTime="2026-07-10 23:00:11.493407427 +0000 UTC m=+272.380143568"
Jul 10 23:00:11.508539 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:11.508467    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bxm84" podStartSLOduration=1.297797614 podStartE2EDuration="4.508445815s" totalImagesPullingTime="3.210648201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:00:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:00:11.507165952 +0000 UTC m=+272.393902096" watchObservedRunningTime="2026-07-10 23:00:11.508445815 +0000 UTC m=+272.395181940"
Jul 10 23:00:11.526869 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:11.526789    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tf99c" podStartSLOduration=1.300229099 podStartE2EDuration="4.526765112s" totalImagesPullingTime="3.226536013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:00:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:00:11.525103123 +0000 UTC m=+272.411839272" watchObservedRunningTime="2026-07-10 23:00:11.526765112 +0000 UTC m=+272.413501256"
Jul 10 23:00:11.540087 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:11.540032    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-24jvw" podStartSLOduration=1.339090327 podStartE2EDuration="4.540011695s" totalImagesPullingTime="3.200921368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:00:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:00:11.538862054 +0000 UTC m=+272.425598212" watchObservedRunningTime="2026-07-10 23:00:11.540011695 +0000 UTC m=+272.426747839"
Jul 10 23:00:11.652320 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:11.652285    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:11.752375 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:11.752269    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:12.259235 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.259195    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:12.344969 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.344927    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:12.348585 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.348551    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-kgdvg"]
Jul 10 23:00:12.352912 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.352886    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-kgdvg"
Jul 10 23:00:12.355225 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.355200    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 10 23:00:12.356148 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.356128    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 10 23:00:12.356276 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.356130    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-v8g5d\""
Jul 10 23:00:12.361189 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.361108    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-kgdvg"]
Jul 10 23:00:12.422787 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.422754    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8c7sr"]
Jul 10 23:00:12.423084 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.422865    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:12.426761 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.426733    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8c7sr"
Jul 10 23:00:12.429488 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.429462    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 10 23:00:12.429785 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.429763    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 10 23:00:12.429896 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.429869    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q2bn6\""
Jul 10 23:00:12.441337 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.441307    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8c7sr"]
Jul 10 23:00:12.474461 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.474428    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49cpr\" (UniqueName: \"kubernetes.io/projected/79faa6c0-9a5c-47b4-b78c-c81181096ba6-kube-api-access-49cpr\") pod \"migrator-7f59b54f6c-kgdvg\" (UID: \"79faa6c0-9a5c-47b4-b78c-c81181096ba6\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-kgdvg"
Jul 10 23:00:12.555000 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.554922    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:12.575052 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.575013    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-data-volume\") pod \"insights-runtime-extractor-8c7sr\" (UID: \"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d\") " pod="openshift-insights/insights-runtime-extractor-8c7sr"
Jul 10 23:00:12.575230 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.575105    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49cpr\" (UniqueName: \"kubernetes.io/projected/79faa6c0-9a5c-47b4-b78c-c81181096ba6-kube-api-access-49cpr\") pod \"migrator-7f59b54f6c-kgdvg\" (UID: \"79faa6c0-9a5c-47b4-b78c-c81181096ba6\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-kgdvg"
Jul 10 23:00:12.575230 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.575155    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-crio-socket\") pod \"insights-runtime-extractor-8c7sr\" (UID: \"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d\") " pod="openshift-insights/insights-runtime-extractor-8c7sr"
Jul 10 23:00:12.575230 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.575182    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8c7sr\" (UID: \"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d\") " pod="openshift-insights/insights-runtime-extractor-8c7sr"
Jul 10 23:00:12.575432 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.575242    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8c7sr\" (UID: \"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d\") " pod="openshift-insights/insights-runtime-extractor-8c7sr"
Jul 10 23:00:12.575432 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.575317    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lpc5\" (UniqueName: \"kubernetes.io/projected/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-kube-api-access-5lpc5\") pod \"insights-runtime-extractor-8c7sr\" (UID: \"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d\") " pod="openshift-insights/insights-runtime-extractor-8c7sr"
Jul 10 23:00:12.589172 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.589145    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49cpr\" (UniqueName: \"kubernetes.io/projected/79faa6c0-9a5c-47b4-b78c-c81181096ba6-kube-api-access-49cpr\") pod \"migrator-7f59b54f6c-kgdvg\" (UID: \"79faa6c0-9a5c-47b4-b78c-c81181096ba6\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-kgdvg"
Jul 10 23:00:12.664477 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.664422    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-kgdvg"
Jul 10 23:00:12.672298 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.672264    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:12.676488 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.676455    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lpc5\" (UniqueName: \"kubernetes.io/projected/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-kube-api-access-5lpc5\") pod \"insights-runtime-extractor-8c7sr\" (UID: \"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d\") " pod="openshift-insights/insights-runtime-extractor-8c7sr"
Jul 10 23:00:12.676628 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.676519    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-data-volume\") pod \"insights-runtime-extractor-8c7sr\" (UID: \"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d\") " pod="openshift-insights/insights-runtime-extractor-8c7sr"
Jul 10 23:00:12.676720 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.676702    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-crio-socket\") pod \"insights-runtime-extractor-8c7sr\" (UID: \"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d\") " pod="openshift-insights/insights-runtime-extractor-8c7sr"
Jul 10 23:00:12.676779 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.676752    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8c7sr\" (UID: \"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d\") " pod="openshift-insights/insights-runtime-extractor-8c7sr"
Jul 10 23:00:12.677117 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.677093    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-crio-socket\") pod \"insights-runtime-extractor-8c7sr\" (UID: \"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d\") " pod="openshift-insights/insights-runtime-extractor-8c7sr"
Jul 10 23:00:12.677249 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.677116    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-data-volume\") pod \"insights-runtime-extractor-8c7sr\" (UID: \"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d\") " pod="openshift-insights/insights-runtime-extractor-8c7sr"
Jul 10 23:00:12.677349 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.677151    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8c7sr\" (UID: \"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d\") " pod="openshift-insights/insights-runtime-extractor-8c7sr"
Jul 10 23:00:12.677477 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:12.677205    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 23:00:12.677477 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:12.677267    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 23:00:12.677568 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:12.677500    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-insights-runtime-extractor-tls podName:67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d nodeName:}" failed. No retries permitted until 2026-07-10 23:00:13.177480535 +0000 UTC m=+274.064216674 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8c7sr" (UID: "67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d") : secret "insights-runtime-extractor-tls" not found
Jul 10 23:00:12.677568 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:12.677544    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-kube-rbac-proxy-cm podName:67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d nodeName:}" failed. No retries permitted until 2026-07-10 23:00:13.177511218 +0000 UTC m=+274.064247360 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8c7sr" (UID: "67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d") : configmap "kube-rbac-proxy" not found
Jul 10 23:00:12.688884 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.688848    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lpc5\" (UniqueName: \"kubernetes.io/projected/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-kube-api-access-5lpc5\") pod \"insights-runtime-extractor-8c7sr\" (UID: \"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d\") " pod="openshift-insights/insights-runtime-extractor-8c7sr"
Jul 10 23:00:12.798164 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.798128    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-kgdvg"]
Jul 10 23:00:12.801723 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:00:12.801685    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79faa6c0_9a5c_47b4_b78c_c81181096ba6.slice/crio-9cafef3caf8be2afdcddad347cef9ee9797f6f8020ee0a5d727f8cbb532fba88 WatchSource:0}: Error finding container 9cafef3caf8be2afdcddad347cef9ee9797f6f8020ee0a5d727f8cbb532fba88: Status 404 returned error can't find the container with id 9cafef3caf8be2afdcddad347cef9ee9797f6f8020ee0a5d727f8cbb532fba88
Jul 10 23:00:12.969350 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:12.969315    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:13.069796 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:13.069710    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:13.182617 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:13.182571    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8c7sr\" (UID: \"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d\") " pod="openshift-insights/insights-runtime-extractor-8c7sr"
Jul 10 23:00:13.182771 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:13.182639    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8c7sr\" (UID: \"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d\") " pod="openshift-insights/insights-runtime-extractor-8c7sr"
Jul 10 23:00:13.182771 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:13.182720    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 23:00:13.182771 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:13.182736    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 23:00:13.182898 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:13.182784    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-kube-rbac-proxy-cm podName:67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d nodeName:}" failed. No retries permitted until 2026-07-10 23:00:14.182770227 +0000 UTC m=+275.069506349 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8c7sr" (UID: "67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d") : configmap "kube-rbac-proxy" not found
Jul 10 23:00:13.182898 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:13.182798    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-insights-runtime-extractor-tls podName:67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d nodeName:}" failed. No retries permitted until 2026-07-10 23:00:14.182791085 +0000 UTC m=+275.069527206 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8c7sr" (UID: "67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d") : secret "insights-runtime-extractor-tls" not found
Jul 10 23:00:13.491334 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:13.491296    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-kgdvg" event={"ID":"79faa6c0-9a5c-47b4-b78c-c81181096ba6","Type":"ContainerStarted","Data":"9cafef3caf8be2afdcddad347cef9ee9797f6f8020ee0a5d727f8cbb532fba88"}
Jul 10 23:00:14.188823 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:14.188786    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8c7sr\" (UID: \"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d\") " pod="openshift-insights/insights-runtime-extractor-8c7sr"
Jul 10 23:00:14.189017 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:14.188880    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8c7sr\" (UID: \"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d\") " pod="openshift-insights/insights-runtime-extractor-8c7sr"
Jul 10 23:00:14.189017 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:14.188954    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 23:00:14.189137 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:14.189011    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 23:00:14.189137 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:14.189036    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-insights-runtime-extractor-tls podName:67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d nodeName:}" failed. No retries permitted until 2026-07-10 23:00:16.189018244 +0000 UTC m=+277.075754372 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8c7sr" (UID: "67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d") : secret "insights-runtime-extractor-tls" not found
Jul 10 23:00:14.189137 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:14.189072    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-kube-rbac-proxy-cm podName:67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d nodeName:}" failed. No retries permitted until 2026-07-10 23:00:16.18905465 +0000 UTC m=+277.075790787 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8c7sr" (UID: "67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d") : configmap "kube-rbac-proxy" not found
Jul 10 23:00:14.567844 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:14.567821    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-qzd5x"]
Jul 10 23:00:14.570738 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:14.570715    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-qzd5x"
Jul 10 23:00:14.572966 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:14.572944    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 10 23:00:14.573844 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:14.573822    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 10 23:00:14.573942 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:14.573823    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-d5pbh\""
Jul 10 23:00:14.573942 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:14.573827    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 10 23:00:14.573942 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:14.573831    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 10 23:00:14.581687 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:14.580554    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-qzd5x"]
Jul 10 23:00:14.592614 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:14.592582    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c91272d2-130f-4578-8a62-b0f8349d6224-signing-cabundle\") pod \"service-ca-79f68b9f4f-qzd5x\" (UID: \"c91272d2-130f-4578-8a62-b0f8349d6224\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qzd5x"
Jul 10 23:00:14.592743 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:14.592623    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lk296\" (UniqueName: \"kubernetes.io/projected/c91272d2-130f-4578-8a62-b0f8349d6224-kube-api-access-lk296\") pod \"service-ca-79f68b9f4f-qzd5x\" (UID: \"c91272d2-130f-4578-8a62-b0f8349d6224\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qzd5x"
Jul 10 23:00:14.592812 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:14.592769    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c91272d2-130f-4578-8a62-b0f8349d6224-signing-key\") pod \"service-ca-79f68b9f4f-qzd5x\" (UID: \"c91272d2-130f-4578-8a62-b0f8349d6224\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qzd5x"
Jul 10 23:00:14.651678 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:14.651642    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:14.693892 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:14.693855    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c91272d2-130f-4578-8a62-b0f8349d6224-signing-key\") pod \"service-ca-79f68b9f4f-qzd5x\" (UID: \"c91272d2-130f-4578-8a62-b0f8349d6224\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qzd5x"
Jul 10 23:00:14.694066 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:14.693928    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c91272d2-130f-4578-8a62-b0f8349d6224-signing-cabundle\") pod \"service-ca-79f68b9f4f-qzd5x\" (UID: \"c91272d2-130f-4578-8a62-b0f8349d6224\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qzd5x"
Jul 10 23:00:14.694066 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:14.693957    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lk296\" (UniqueName: \"kubernetes.io/projected/c91272d2-130f-4578-8a62-b0f8349d6224-kube-api-access-lk296\") pod \"service-ca-79f68b9f4f-qzd5x\" (UID: \"c91272d2-130f-4578-8a62-b0f8349d6224\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qzd5x"
Jul 10 23:00:14.694802 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:14.694778    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c91272d2-130f-4578-8a62-b0f8349d6224-signing-cabundle\") pod \"service-ca-79f68b9f4f-qzd5x\" (UID: \"c91272d2-130f-4578-8a62-b0f8349d6224\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qzd5x"
Jul 10 23:00:14.696657 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:14.696632    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c91272d2-130f-4578-8a62-b0f8349d6224-signing-key\") pod \"service-ca-79f68b9f4f-qzd5x\" (UID: \"c91272d2-130f-4578-8a62-b0f8349d6224\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qzd5x"
Jul 10 23:00:14.702314 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:14.702289    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lk296\" (UniqueName: \"kubernetes.io/projected/c91272d2-130f-4578-8a62-b0f8349d6224-kube-api-access-lk296\") pod \"service-ca-79f68b9f4f-qzd5x\" (UID: \"c91272d2-130f-4578-8a62-b0f8349d6224\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qzd5x"
Jul 10 23:00:14.750769 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:14.750679    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:14.899866 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:14.899823    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-qzd5x"
Jul 10 23:00:14.997229 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:14.997176    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dae10db7-61ca-4e1e-80f2-ab4fb81ebde2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-9tbg6\" (UID: \"dae10db7-61ca-4e1e-80f2-ab4fb81ebde2\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9tbg6"
Jul 10 23:00:14.997435 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:14.997347    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 10 23:00:14.997501 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:14.997449    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dae10db7-61ca-4e1e-80f2-ab4fb81ebde2-cluster-monitoring-operator-tls podName:dae10db7-61ca-4e1e-80f2-ab4fb81ebde2 nodeName:}" failed. No retries permitted until 2026-07-10 23:00:22.997423561 +0000 UTC m=+283.884159699 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/dae10db7-61ca-4e1e-80f2-ab4fb81ebde2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-9tbg6" (UID: "dae10db7-61ca-4e1e-80f2-ab4fb81ebde2") : secret "cluster-monitoring-operator-tls" not found
Jul 10 23:00:15.032323 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:15.032241    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-qzd5x"]
Jul 10 23:00:15.035516 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:00:15.035486    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc91272d2_130f_4578_8a62_b0f8349d6224.slice/crio-01e169afbb115c6c9c2f3b706652f69a99d7e4c71af1234897ad1e729fa2ab15 WatchSource:0}: Error finding container 01e169afbb115c6c9c2f3b706652f69a99d7e4c71af1234897ad1e729fa2ab15: Status 404 returned error can't find the container with id 01e169afbb115c6c9c2f3b706652f69a99d7e4c71af1234897ad1e729fa2ab15
Jul 10 23:00:15.434106 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:15.434021    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kd62p_54d84e33-5ba4-4b6c-8c8f-4ddaac24eb03/dns-node-resolver/0.log"
Jul 10 23:00:15.499986 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:15.499946    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-qzd5x" event={"ID":"c91272d2-130f-4578-8a62-b0f8349d6224","Type":"ContainerStarted","Data":"4fe7f2b0bb5806f2abbc4089e9b75bcb08b1eff7ecce25817c4b98329792831d"}
Jul 10 23:00:15.500157 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:15.500010    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-qzd5x" event={"ID":"c91272d2-130f-4578-8a62-b0f8349d6224","Type":"ContainerStarted","Data":"01e169afbb115c6c9c2f3b706652f69a99d7e4c71af1234897ad1e729fa2ab15"}
Jul 10 23:00:15.501936 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:15.501904    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-kgdvg" event={"ID":"79faa6c0-9a5c-47b4-b78c-c81181096ba6","Type":"ContainerStarted","Data":"b051146e0d29f32486c198529a21022ea91d8bbe212e10f7b744156bbc526313"}
Jul 10 23:00:15.502050 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:15.501963    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-kgdvg" event={"ID":"79faa6c0-9a5c-47b4-b78c-c81181096ba6","Type":"ContainerStarted","Data":"296ec16cdebff343bf9001d5b1234f30a8c5d9cbe625743d514d68501a7eddad"}
Jul 10 23:00:15.517344 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:15.517291    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-qzd5x" podStartSLOduration=1.517275215 podStartE2EDuration="1.517275215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:00:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:00:15.516533626 +0000 UTC m=+276.403269792" watchObservedRunningTime="2026-07-10 23:00:15.517275215 +0000 UTC m=+276.404011358"
Jul 10 23:00:15.533731 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:15.533674    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-kgdvg" podStartSLOduration=1.857034517 podStartE2EDuration="3.533657862s" totalImagesPullingTime="1.676623345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:00:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:00:15.532729949 +0000 UTC m=+276.419466093" watchObservedRunningTime="2026-07-10 23:00:15.533657862 +0000 UTC m=+276.420394005"
Jul 10 23:00:15.578648 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:15.578597    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:15.889660 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:15.889622    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:16.029869 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:16.029834    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-298xj_e17cacf8-0106-484f-a226-2f584a8fc5a5/node-ca/0.log"
Jul 10 23:00:16.207649 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:16.207550    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8c7sr\" (UID: \"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d\") " pod="openshift-insights/insights-runtime-extractor-8c7sr"
Jul 10 23:00:16.207649 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:16.207636    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8c7sr\" (UID: \"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d\") " pod="openshift-insights/insights-runtime-extractor-8c7sr"
Jul 10 23:00:16.207884 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:16.207720    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 23:00:16.207884 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:16.207776    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 23:00:16.207884 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:16.207798    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-insights-runtime-extractor-tls podName:67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d nodeName:}" failed. No retries permitted until 2026-07-10 23:00:20.207777028 +0000 UTC m=+281.094513167 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8c7sr" (UID: "67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d") : secret "insights-runtime-extractor-tls" not found
Jul 10 23:00:16.207884 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:16.207830    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-kube-rbac-proxy-cm podName:67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d nodeName:}" failed. No retries permitted until 2026-07-10 23:00:20.207812331 +0000 UTC m=+281.094548458 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8c7sr" (UID: "67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d") : configmap "kube-rbac-proxy" not found
Jul 10 23:00:16.449424 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:16.449383    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:16.549573 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:16.549535    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:16.615474 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:16.615441    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:17.030051 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:17.030023    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9ttw7_67dc0695-fad8-4ecf-bc13-3a7410c1b195/serve-healthcheck-canary/0.log"
Jul 10 23:00:17.430468 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:17.430393    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-kgdvg_79faa6c0-9a5c-47b4-b78c-c81181096ba6/migrator/0.log"
Jul 10 23:00:17.575089 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:17.575055    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:17.629987 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:17.629959    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-kgdvg_79faa6c0-9a5c-47b4-b78c-c81181096ba6/graceful-termination/0.log"
Jul 10 23:00:17.830587 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:17.830545    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-bxm84_5dd8a036-d481-4d56-9e10-1dc8e52983ac/kube-storage-version-migrator-operator/0.log"
Jul 10 23:00:18.083907 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:18.083810    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:18.183218 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:18.183188    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:18.373204 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:18.373114    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:18.576504 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:18.576472    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:18.767602 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:18.767566    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:19.375772 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:19.375737    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:19.481047 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:19.481018    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:19.781034 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:19.780945    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:19.970852 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:19.970815    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:20.239907 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:20.239871    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8c7sr\" (UID: \"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d\") " pod="openshift-insights/insights-runtime-extractor-8c7sr"
Jul 10 23:00:20.240063 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:20.239946    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8c7sr\" (UID: \"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d\") " pod="openshift-insights/insights-runtime-extractor-8c7sr"
Jul 10 23:00:20.240063 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:20.240031    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 23:00:20.240063 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:20.240042    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 23:00:20.240174 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:20.240098    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-insights-runtime-extractor-tls podName:67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d nodeName:}" failed. No retries permitted until 2026-07-10 23:00:28.240082031 +0000 UTC m=+289.126818151 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8c7sr" (UID: "67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d") : secret "insights-runtime-extractor-tls" not found
Jul 10 23:00:20.240174 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:20.240115    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-kube-rbac-proxy-cm podName:67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d nodeName:}" failed. No retries permitted until 2026-07-10 23:00:28.240107665 +0000 UTC m=+289.126843785 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8c7sr" (UID: "67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d") : configmap "kube-rbac-proxy" not found
Jul 10 23:00:20.648248 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:20.648164    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:20.766239 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:20.766208    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:21.199749 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:21.199699    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-v2p2h" podUID="741d147a-735b-40ef-a55b-8e8f03b062d0"
Jul 10 23:00:21.199749 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:21.199699    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-769cc77955-d2747" podUID="e7f0af82-786b-4b48-be84-9ced1047ff52"
Jul 10 23:00:21.200168 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:21.199708    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cf856758c-l9t9h" podUID="2981ee03-5541-484c-8226-2d606103d31e"
Jul 10 23:00:21.369065 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:21.369032    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:21.520894 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:21.520862    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-l9t9h"
Jul 10 23:00:21.521083 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:21.520867    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v2p2h"
Jul 10 23:00:21.521083 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:21.520866    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 23:00:21.759078 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:21.759042    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:22.049829 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:22.049796    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:22.514110 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:22.514079    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:22.615252 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:22.615220    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:22.968080 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:22.967996    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:23.064214 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:23.064151    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dae10db7-61ca-4e1e-80f2-ab4fb81ebde2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-9tbg6\" (UID: \"dae10db7-61ca-4e1e-80f2-ab4fb81ebde2\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9tbg6"
Jul 10 23:00:23.066999 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:23.066964    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dae10db7-61ca-4e1e-80f2-ab4fb81ebde2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-9tbg6\" (UID: \"dae10db7-61ca-4e1e-80f2-ab4fb81ebde2\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9tbg6"
Jul 10 23:00:23.108784 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:23.108737    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9tbg6"
Jul 10 23:00:23.166641 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:23.166467    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:23.255840 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:23.255714    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9tbg6"]
Jul 10 23:00:23.258936 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:00:23.258909    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddae10db7_61ca_4e1e_80f2_ab4fb81ebde2.slice/crio-0e8a92e2180445240eaaf6ea52ac40b7699b33a53b091d587f6558be32adf80d WatchSource:0}: Error finding container 0e8a92e2180445240eaaf6ea52ac40b7699b33a53b091d587f6558be32adf80d: Status 404 returned error can't find the container with id 0e8a92e2180445240eaaf6ea52ac40b7699b33a53b091d587f6558be32adf80d
Jul 10 23:00:23.529907 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:23.529822    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9tbg6" event={"ID":"dae10db7-61ca-4e1e-80f2-ab4fb81ebde2","Type":"ContainerStarted","Data":"0e8a92e2180445240eaaf6ea52ac40b7699b33a53b091d587f6558be32adf80d"}
Jul 10 23:00:23.759299 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:23.759258    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:23.964500 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:23.964409    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:24.363056 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:24.363022    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:24.580776 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:24.580735    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls\") pod \"dns-default-v2p2h\" (UID: \"741d147a-735b-40ef-a55b-8e8f03b062d0\") " pod="openshift-dns/dns-default-v2p2h"
Jul 10 23:00:24.581242 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:24.580798    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-l9t9h\" (UID: \"2981ee03-5541-484c-8226-2d606103d31e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-l9t9h"
Jul 10 23:00:24.581242 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:24.580856    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 23:00:24.583652 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:24.583624    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/741d147a-735b-40ef-a55b-8e8f03b062d0-metrics-tls\") pod \"dns-default-v2p2h\" (UID: \"741d147a-735b-40ef-a55b-8e8f03b062d0\") " pod="openshift-dns/dns-default-v2p2h"
Jul 10 23:00:24.583812 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:24.583774    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls\") pod \"image-registry-769cc77955-d2747\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") " pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 23:00:24.584039 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:24.584020    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2981ee03-5541-484c-8226-2d606103d31e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-l9t9h\" (UID: \"2981ee03-5541-484c-8226-2d606103d31e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-l9t9h"
Jul 10 23:00:24.824940 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:24.824907    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-xxpgj\""
Jul 10 23:00:24.825150 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:24.825012    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-67lgp\""
Jul 10 23:00:24.825150 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:24.825017    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-j5n6l\""
Jul 10 23:00:24.832059 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:24.832022    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 23:00:24.832059 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:24.832057    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v2p2h"
Jul 10 23:00:24.832289 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:24.832025    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-l9t9h"
Jul 10 23:00:25.164024 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:25.163990    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:25.171692 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:25.171560    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-v2p2h"]
Jul 10 23:00:25.174874 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:00:25.174845    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod741d147a_735b_40ef_a55b_8e8f03b062d0.slice/crio-9d9a6318952fe0bcc4bae746de289d767df54dd91a0d6b59a4729faad5e08c16 WatchSource:0}: Error finding container 9d9a6318952fe0bcc4bae746de289d767df54dd91a0d6b59a4729faad5e08c16: Status 404 returned error can't find the container with id 9d9a6318952fe0bcc4bae746de289d767df54dd91a0d6b59a4729faad5e08c16
Jul 10 23:00:25.361853 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:25.361774    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:25.402914 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:25.399874    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-l9t9h"]
Jul 10 23:00:25.403088 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:00:25.402980    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2981ee03_5541_484c_8226_2d606103d31e.slice/crio-e5b0382313b48faa5137372669ad9dfdfdfa7e1ce9be2c04993137afcd734646 WatchSource:0}: Error finding container e5b0382313b48faa5137372669ad9dfdfdfa7e1ce9be2c04993137afcd734646: Status 404 returned error can't find the container with id e5b0382313b48faa5137372669ad9dfdfdfa7e1ce9be2c04993137afcd734646
Jul 10 23:00:25.405542 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:25.405515    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-769cc77955-d2747"]
Jul 10 23:00:25.408507 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:00:25.408484    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7f0af82_786b_4b48_be84_9ced1047ff52.slice/crio-668d108ae8cbe4975c8bfb44d736090536b1d7fcc69b616bda079a7e630ca03b WatchSource:0}: Error finding container 668d108ae8cbe4975c8bfb44d736090536b1d7fcc69b616bda079a7e630ca03b: Status 404 returned error can't find the container with id 668d108ae8cbe4975c8bfb44d736090536b1d7fcc69b616bda079a7e630ca03b
Jul 10 23:00:25.471216 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:25.471181    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:25.537870 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:25.537831    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9tbg6" event={"ID":"dae10db7-61ca-4e1e-80f2-ab4fb81ebde2","Type":"ContainerStarted","Data":"0290c33e443924c1f60495daa7d13556fa458a2a26c810e4f503e6c3a35bed88"}
Jul 10 23:00:25.539297 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:25.539267    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-l9t9h" event={"ID":"2981ee03-5541-484c-8226-2d606103d31e","Type":"ContainerStarted","Data":"e5b0382313b48faa5137372669ad9dfdfdfa7e1ce9be2c04993137afcd734646"}
Jul 10 23:00:25.540557 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:25.540529    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v2p2h" event={"ID":"741d147a-735b-40ef-a55b-8e8f03b062d0","Type":"ContainerStarted","Data":"9d9a6318952fe0bcc4bae746de289d767df54dd91a0d6b59a4729faad5e08c16"}
Jul 10 23:00:25.542121 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:25.542096    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-769cc77955-d2747" event={"ID":"e7f0af82-786b-4b48-be84-9ced1047ff52","Type":"ContainerStarted","Data":"c995a3e52edb0293b3a881b7a5d8379fb9f58495ffe24be02697e297634b1a86"}
Jul 10 23:00:25.542219 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:25.542138    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-769cc77955-d2747" event={"ID":"e7f0af82-786b-4b48-be84-9ced1047ff52","Type":"ContainerStarted","Data":"668d108ae8cbe4975c8bfb44d736090536b1d7fcc69b616bda079a7e630ca03b"}
Jul 10 23:00:25.542267 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:25.542250    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 23:00:25.555315 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:25.555256    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9tbg6" podStartSLOduration=16.748574781 podStartE2EDuration="18.555236181s" totalImagesPullingTime="1.8066614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:00:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:00:25.554381121 +0000 UTC m=+286.441117260" watchObservedRunningTime="2026-07-10 23:00:25.555236181 +0000 UTC m=+286.441972327"
Jul 10 23:00:25.559339 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:25.559309    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:25.575701 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:25.575638    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-769cc77955-d2747" podStartSLOduration=285.575613446 podStartE2EDuration="4m45.575613446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:55:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:00:25.574289891 +0000 UTC m=+286.461026033" watchObservedRunningTime="2026-07-10 23:00:25.575613446 +0000 UTC m=+286.462349586"
Jul 10 23:00:25.628517 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:25.628397    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:25.836701 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:25.836658    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:25.963295 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:25.963217    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:26.384030 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:26.383991    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:27.056294 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:27.056255    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:27.118404 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:27.118346    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:27.555023 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:27.554989    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-v2p2h"
Jul 10 23:00:27.555188 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:27.555042    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-l9t9h" event={"ID":"2981ee03-5541-484c-8226-2d606103d31e","Type":"ContainerStarted","Data":"dba1e0e89271463bc3ee64238fe11481f478cc29a107b2a824c657ce508b7d3c"}
Jul 10 23:00:27.555188 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:27.555062    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v2p2h" event={"ID":"741d147a-735b-40ef-a55b-8e8f03b062d0","Type":"ContainerStarted","Data":"dcb4c4520f431e03cff95263abd5dfdee1fbb7eb32af383f54cbcb0cc75a295d"}
Jul 10 23:00:27.555188 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:27.555082    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v2p2h" event={"ID":"741d147a-735b-40ef-a55b-8e8f03b062d0","Type":"ContainerStarted","Data":"f24031563ff7c3a051b8306a8d4200ead580e938fc60279bcce77749b87dfaed"}
Jul 10 23:00:27.570305 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:27.570205    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-l9t9h" podStartSLOduration=283.6739261 podStartE2EDuration="4m45.570191739s" totalImagesPullingTime="1.896265639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:55:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:00:27.568292082 +0000 UTC m=+288.455028225" watchObservedRunningTime="2026-07-10 23:00:27.570191739 +0000 UTC m=+288.456927881"
Jul 10 23:00:27.586755 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:27.586702    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-v2p2h" podStartSLOduration=251.457312734 podStartE2EDuration="4m13.586684096s" totalImagesPullingTime="2.129371362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:56:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:00:27.586436702 +0000 UTC m=+288.473172860" watchObservedRunningTime="2026-07-10 23:00:27.586684096 +0000 UTC m=+288.473420239"
Jul 10 23:00:28.314682 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:28.314637    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8c7sr\" (UID: \"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d\") " pod="openshift-insights/insights-runtime-extractor-8c7sr"
Jul 10 23:00:28.315164 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:28.314695    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8c7sr\" (UID: \"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d\") " pod="openshift-insights/insights-runtime-extractor-8c7sr"
Jul 10 23:00:28.315230 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:28.315177    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8c7sr\" (UID: \"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d\") " pod="openshift-insights/insights-runtime-extractor-8c7sr"
Jul 10 23:00:28.317235 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:28.317210    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8c7sr\" (UID: \"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d\") " pod="openshift-insights/insights-runtime-extractor-8c7sr"
Jul 10 23:00:28.337981 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:28.337937    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8c7sr"
Jul 10 23:00:28.481709 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:28.481677    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8c7sr"]
Jul 10 23:00:28.485287 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:00:28.485261    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67b7ba69_2f4d_49f9_a2e1_893d5ebfee7d.slice/crio-14670e4249469c206d192f2d20b3f914169b85fa4da5d9a44393805adccbfff9 WatchSource:0}: Error finding container 14670e4249469c206d192f2d20b3f914169b85fa4da5d9a44393805adccbfff9: Status 404 returned error can't find the container with id 14670e4249469c206d192f2d20b3f914169b85fa4da5d9a44393805adccbfff9
Jul 10 23:00:28.559129 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:28.559091    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8c7sr" event={"ID":"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d","Type":"ContainerStarted","Data":"14670e4249469c206d192f2d20b3f914169b85fa4da5d9a44393805adccbfff9"}
Jul 10 23:00:31.264557 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:31.264512    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:31.366833 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:31.366800    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:31.462983 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:31.462941    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:31.747380 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:31.747304    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:31.805875 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:31.805831    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:31.937089 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:31.937057    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:31.987443 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:31.987404    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:32.367704 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:32.367665    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:32.423479 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:32.423441    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:32.942625 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:32.942586    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:33.044293 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:33.044254    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:33.146506 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:33.146473    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:33.563197 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:33.563164    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:33.579145 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:33.579110    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8c7sr" event={"ID":"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d","Type":"ContainerStarted","Data":"24394ffbe7db11cd20675db0292be51ad0fec432db28a39ca4ef3fa0bf22916d"}
Jul 10 23:00:33.579307 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:33.579173    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8c7sr" event={"ID":"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d","Type":"ContainerStarted","Data":"23e4af07a02c32b5eb136ba2fb7d49dfe1fc32f5b0884d092fa40a4c8c8e1318"}
Jul 10 23:00:33.663246 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:33.663217    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:35.551007 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:35.550959    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-w9kxw" podUID="9829a27f-3dbd-44c7-b278-f26beb69f82a"
Jul 10 23:00:36.593049 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:36.592953    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8c7sr" event={"ID":"67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d","Type":"ContainerStarted","Data":"2ab1a4b4cbf4006f10a978f299d5af5b231dc83efa911910f050a8f432e6ba5f"}
Jul 10 23:00:36.610998 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:36.610949    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8c7sr" podStartSLOduration=16.921973376 podStartE2EDuration="24.610933393s" totalImagesPullingTime="7.688960017s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 23:00:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:00:36.610625678 +0000 UTC m=+297.497361821" watchObservedRunningTime="2026-07-10 23:00:36.610933393 +0000 UTC m=+297.497669535"
Jul 10 23:00:37.561515 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:37.561482    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-v2p2h"
Jul 10 23:00:38.627297 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:38.627261    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:38.648698 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:38.648659    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-945ss"]
Jul 10 23:00:38.652052 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:38.652024    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-945ss"
Jul 10 23:00:38.654488 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:38.654460    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 10 23:00:38.654621 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:38.654512    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-dq7mr\""
Jul 10 23:00:38.654621 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:38.654528    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 10 23:00:38.667501 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:38.667467    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-945ss"]
Jul 10 23:00:38.770014 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:38.769967    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:38.804723 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:38.804683    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzqcc\" (UniqueName: \"kubernetes.io/projected/24c76975-a76f-48e0-b1a8-098242d58511-kube-api-access-jzqcc\") pod \"downloads-68c9cbb497-945ss\" (UID: \"24c76975-a76f-48e0-b1a8-098242d58511\") " pod="openshift-console/downloads-68c9cbb497-945ss"
Jul 10 23:00:38.871585 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:38.871552    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:38.905861 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:38.905765    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzqcc\" (UniqueName: \"kubernetes.io/projected/24c76975-a76f-48e0-b1a8-098242d58511-kube-api-access-jzqcc\") pod \"downloads-68c9cbb497-945ss\" (UID: \"24c76975-a76f-48e0-b1a8-098242d58511\") " pod="openshift-console/downloads-68c9cbb497-945ss"
Jul 10 23:00:38.913948 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:38.913908    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzqcc\" (UniqueName: \"kubernetes.io/projected/24c76975-a76f-48e0-b1a8-098242d58511-kube-api-access-jzqcc\") pod \"downloads-68c9cbb497-945ss\" (UID: \"24c76975-a76f-48e0-b1a8-098242d58511\") " pod="openshift-console/downloads-68c9cbb497-945ss"
Jul 10 23:00:38.961857 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:38.961812    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-945ss"
Jul 10 23:00:39.096811 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:39.096769    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-945ss"]
Jul 10 23:00:39.100475 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:00:39.100442    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24c76975_a76f_48e0_b1a8_098242d58511.slice/crio-de5a4b373b9a01ba239682efa155f83fd581d04559c6c964fa9b4ea0696db600 WatchSource:0}: Error finding container de5a4b373b9a01ba239682efa155f83fd581d04559c6c964fa9b4ea0696db600: Status 404 returned error can't find the container with id de5a4b373b9a01ba239682efa155f83fd581d04559c6c964fa9b4ea0696db600
Jul 10 23:00:39.435045 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:39.435019    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/ovn-acl-logging/0.log"
Jul 10 23:00:39.435750 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:39.435729    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/ovn-acl-logging/0.log"
Jul 10 23:00:39.438406 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:39.438384    2563 kubelet.go:1648] "Image garbage collection succeeded"
Jul 10 23:00:39.603802 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:39.603648    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-945ss" event={"ID":"24c76975-a76f-48e0-b1a8-098242d58511","Type":"ContainerStarted","Data":"de5a4b373b9a01ba239682efa155f83fd581d04559c6c964fa9b4ea0696db600"}
Jul 10 23:00:41.659280 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:41.659244    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:41.783622 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:41.783583    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77758c754-58r8c"]
Jul 10 23:00:41.787020 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:41.786990    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77758c754-58r8c"
Jul 10 23:00:41.790459 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:41.790428    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 10 23:00:41.790613 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:41.790562    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 10 23:00:41.790613 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:41.790582    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-kbq4z\""
Jul 10 23:00:41.790613 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:41.790586    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 10 23:00:41.790766 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:41.790693    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 10 23:00:41.790816 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:41.790783    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 10 23:00:41.795126 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:41.795095    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77758c754-58r8c"]
Jul 10 23:00:41.932461 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:41.932350    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-console-oauth-config\") pod \"console-77758c754-58r8c\" (UID: \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\") " pod="openshift-console/console-77758c754-58r8c"
Jul 10 23:00:41.932636 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:41.932479    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-service-ca\") pod \"console-77758c754-58r8c\" (UID: \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\") " pod="openshift-console/console-77758c754-58r8c"
Jul 10 23:00:41.932636 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:41.932512    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn79m\" (UniqueName: \"kubernetes.io/projected/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-kube-api-access-dn79m\") pod \"console-77758c754-58r8c\" (UID: \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\") " pod="openshift-console/console-77758c754-58r8c"
Jul 10 23:00:41.932636 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:41.932541    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-console-serving-cert\") pod \"console-77758c754-58r8c\" (UID: \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\") " pod="openshift-console/console-77758c754-58r8c"
Jul 10 23:00:41.932636 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:41.932606    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-oauth-serving-cert\") pod \"console-77758c754-58r8c\" (UID: \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\") " pod="openshift-console/console-77758c754-58r8c"
Jul 10 23:00:41.932836 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:41.932647    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-console-config\") pod \"console-77758c754-58r8c\" (UID: \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\") " pod="openshift-console/console-77758c754-58r8c"
Jul 10 23:00:42.033607 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:42.033563    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-console-oauth-config\") pod \"console-77758c754-58r8c\" (UID: \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\") " pod="openshift-console/console-77758c754-58r8c"
Jul 10 23:00:42.033830 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:42.033636    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-service-ca\") pod \"console-77758c754-58r8c\" (UID: \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\") " pod="openshift-console/console-77758c754-58r8c"
Jul 10 23:00:42.033830 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:42.033666    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn79m\" (UniqueName: \"kubernetes.io/projected/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-kube-api-access-dn79m\") pod \"console-77758c754-58r8c\" (UID: \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\") " pod="openshift-console/console-77758c754-58r8c"
Jul 10 23:00:42.033830 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:42.033692    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-console-serving-cert\") pod \"console-77758c754-58r8c\" (UID: \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\") " pod="openshift-console/console-77758c754-58r8c"
Jul 10 23:00:42.033830 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:42.033719    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-oauth-serving-cert\") pod \"console-77758c754-58r8c\" (UID: \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\") " pod="openshift-console/console-77758c754-58r8c"
Jul 10 23:00:42.033830 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:42.033744    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-console-config\") pod \"console-77758c754-58r8c\" (UID: \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\") " pod="openshift-console/console-77758c754-58r8c"
Jul 10 23:00:42.034669 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:42.034597    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-service-ca\") pod \"console-77758c754-58r8c\" (UID: \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\") " pod="openshift-console/console-77758c754-58r8c"
Jul 10 23:00:42.034669 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:42.034660    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-oauth-serving-cert\") pod \"console-77758c754-58r8c\" (UID: \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\") " pod="openshift-console/console-77758c754-58r8c"
Jul 10 23:00:42.035129 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:42.034964    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-console-config\") pod \"console-77758c754-58r8c\" (UID: \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\") " pod="openshift-console/console-77758c754-58r8c"
Jul 10 23:00:42.036855 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:42.036825    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-console-oauth-config\") pod \"console-77758c754-58r8c\" (UID: \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\") " pod="openshift-console/console-77758c754-58r8c"
Jul 10 23:00:42.037185 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:42.037168    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-console-serving-cert\") pod \"console-77758c754-58r8c\" (UID: \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\") " pod="openshift-console/console-77758c754-58r8c"
Jul 10 23:00:42.048115 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:42.048085    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn79m\" (UniqueName: \"kubernetes.io/projected/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-kube-api-access-dn79m\") pod \"console-77758c754-58r8c\" (UID: \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\") " pod="openshift-console/console-77758c754-58r8c"
Jul 10 23:00:42.098898 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:42.098850    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77758c754-58r8c"
Jul 10 23:00:42.263114 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:42.263073    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77758c754-58r8c"]
Jul 10 23:00:42.266164 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:00:42.266126    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55ee7f77_d691_4e70_a9fc_35cb36f9c56d.slice/crio-a09c967b7efd727c7bf635347608ee4af73d631dd1e82d1b142fbcbdda6a84df WatchSource:0}: Error finding container a09c967b7efd727c7bf635347608ee4af73d631dd1e82d1b142fbcbdda6a84df: Status 404 returned error can't find the container with id a09c967b7efd727c7bf635347608ee4af73d631dd1e82d1b142fbcbdda6a84df
Jul 10 23:00:42.268455 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:42.268431    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 23:00:42.619483 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:42.619441    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77758c754-58r8c" event={"ID":"55ee7f77-d691-4e70-a9fc-35cb36f9c56d","Type":"ContainerStarted","Data":"a09c967b7efd727c7bf635347608ee4af73d631dd1e82d1b142fbcbdda6a84df"}
Jul 10 23:00:43.016835 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:43.016780    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:43.162979 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:43.162941    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:44.808648 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:44.808520    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:44.837677 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:44.837632    2563 patch_prober.go:28] interesting pod/image-registry-769cc77955-d2747 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 10 23:00:44.837864 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:44.837710    2563 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-769cc77955-d2747" podUID="e7f0af82-786b-4b48-be84-9ced1047ff52" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 23:00:46.550938 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:46.550898    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 23:00:46.553399 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:46.553356    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 23:00:46.643641 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:46.643603    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77758c754-58r8c" event={"ID":"55ee7f77-d691-4e70-a9fc-35cb36f9c56d","Type":"ContainerStarted","Data":"86a45fdbb16e17fb819e77d198a97cc4edb791611de089262cb4fa8aa9b38a16"}
Jul 10 23:00:46.661144 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:46.661067    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77758c754-58r8c" podStartSLOduration=2.155003695 podStartE2EDuration="5.661048466s" totalImagesPullingTime="3.506044771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:00:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:00:46.660104029 +0000 UTC m=+307.546840173" watchObservedRunningTime="2026-07-10 23:00:46.661048466 +0000 UTC m=+307.547784589"
Jul 10 23:00:46.843235 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:46.843140    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:46.941693 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:46.941655    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:47.036957 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.036853    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:47.038439 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.038350    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-q8r92"]
Jul 10 23:00:47.042692 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.042670    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.046326 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.046299    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 10 23:00:47.046609 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.046591    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-m25jt\""
Jul 10 23:00:47.046827 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.046809    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 10 23:00:47.047051 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.047032    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 10 23:00:47.047234 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.047218    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 10 23:00:47.122956 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.122859    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:47.184844 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.184804    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/52440bd8-c787-4a9b-99e9-797246119711-node-exporter-wtmp\") pod \"node-exporter-q8r92\" (UID: \"52440bd8-c787-4a9b-99e9-797246119711\") " pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.185049 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.184856    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/52440bd8-c787-4a9b-99e9-797246119711-node-exporter-textfile\") pod \"node-exporter-q8r92\" (UID: \"52440bd8-c787-4a9b-99e9-797246119711\") " pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.185049 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.184936    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52440bd8-c787-4a9b-99e9-797246119711-sys\") pod \"node-exporter-q8r92\" (UID: \"52440bd8-c787-4a9b-99e9-797246119711\") " pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.185049 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.185021    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52440bd8-c787-4a9b-99e9-797246119711-metrics-client-ca\") pod \"node-exporter-q8r92\" (UID: \"52440bd8-c787-4a9b-99e9-797246119711\") " pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.185213 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.185082    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bknbc\" (UniqueName: \"kubernetes.io/projected/52440bd8-c787-4a9b-99e9-797246119711-kube-api-access-bknbc\") pod \"node-exporter-q8r92\" (UID: \"52440bd8-c787-4a9b-99e9-797246119711\") " pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.185213 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.185136    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/52440bd8-c787-4a9b-99e9-797246119711-node-exporter-tls\") pod \"node-exporter-q8r92\" (UID: \"52440bd8-c787-4a9b-99e9-797246119711\") " pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.185213 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.185197    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52440bd8-c787-4a9b-99e9-797246119711-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q8r92\" (UID: \"52440bd8-c787-4a9b-99e9-797246119711\") " pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.185354 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.185244    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/52440bd8-c787-4a9b-99e9-797246119711-node-exporter-accelerators-collector-config\") pod \"node-exporter-q8r92\" (UID: \"52440bd8-c787-4a9b-99e9-797246119711\") " pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.185354 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.185287    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/52440bd8-c787-4a9b-99e9-797246119711-root\") pod \"node-exporter-q8r92\" (UID: \"52440bd8-c787-4a9b-99e9-797246119711\") " pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.222713 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.222662    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:47.286284 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.286235    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/52440bd8-c787-4a9b-99e9-797246119711-node-exporter-textfile\") pod \"node-exporter-q8r92\" (UID: \"52440bd8-c787-4a9b-99e9-797246119711\") " pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.286488 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.286298    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52440bd8-c787-4a9b-99e9-797246119711-sys\") pod \"node-exporter-q8r92\" (UID: \"52440bd8-c787-4a9b-99e9-797246119711\") " pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.286488 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.286349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52440bd8-c787-4a9b-99e9-797246119711-metrics-client-ca\") pod \"node-exporter-q8r92\" (UID: \"52440bd8-c787-4a9b-99e9-797246119711\") " pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.286488 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.286420    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bknbc\" (UniqueName: \"kubernetes.io/projected/52440bd8-c787-4a9b-99e9-797246119711-kube-api-access-bknbc\") pod \"node-exporter-q8r92\" (UID: \"52440bd8-c787-4a9b-99e9-797246119711\") " pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.286488 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.286447    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/52440bd8-c787-4a9b-99e9-797246119711-node-exporter-tls\") pod \"node-exporter-q8r92\" (UID: \"52440bd8-c787-4a9b-99e9-797246119711\") " pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.286488 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.286470    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52440bd8-c787-4a9b-99e9-797246119711-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q8r92\" (UID: \"52440bd8-c787-4a9b-99e9-797246119711\") " pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.286764 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.286496    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/52440bd8-c787-4a9b-99e9-797246119711-node-exporter-accelerators-collector-config\") pod \"node-exporter-q8r92\" (UID: \"52440bd8-c787-4a9b-99e9-797246119711\") " pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.286764 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.286521    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/52440bd8-c787-4a9b-99e9-797246119711-root\") pod \"node-exporter-q8r92\" (UID: \"52440bd8-c787-4a9b-99e9-797246119711\") " pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.286764 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.286562    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/52440bd8-c787-4a9b-99e9-797246119711-node-exporter-wtmp\") pod \"node-exporter-q8r92\" (UID: \"52440bd8-c787-4a9b-99e9-797246119711\") " pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.286764 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.286719    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/52440bd8-c787-4a9b-99e9-797246119711-node-exporter-wtmp\") pod \"node-exporter-q8r92\" (UID: \"52440bd8-c787-4a9b-99e9-797246119711\") " pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.287051 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.287034    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/52440bd8-c787-4a9b-99e9-797246119711-node-exporter-textfile\") pod \"node-exporter-q8r92\" (UID: \"52440bd8-c787-4a9b-99e9-797246119711\") " pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.287127 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.287085    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52440bd8-c787-4a9b-99e9-797246119711-sys\") pod \"node-exporter-q8r92\" (UID: \"52440bd8-c787-4a9b-99e9-797246119711\") " pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.287589 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.287570    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52440bd8-c787-4a9b-99e9-797246119711-metrics-client-ca\") pod \"node-exporter-q8r92\" (UID: \"52440bd8-c787-4a9b-99e9-797246119711\") " pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.289088 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.288699    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/52440bd8-c787-4a9b-99e9-797246119711-node-exporter-accelerators-collector-config\") pod \"node-exporter-q8r92\" (UID: \"52440bd8-c787-4a9b-99e9-797246119711\") " pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.289088 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.288769    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/52440bd8-c787-4a9b-99e9-797246119711-root\") pod \"node-exporter-q8r92\" (UID: \"52440bd8-c787-4a9b-99e9-797246119711\") " pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.292289 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.292241    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/52440bd8-c787-4a9b-99e9-797246119711-node-exporter-tls\") pod \"node-exporter-q8r92\" (UID: \"52440bd8-c787-4a9b-99e9-797246119711\") " pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.294118 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.294071    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52440bd8-c787-4a9b-99e9-797246119711-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q8r92\" (UID: \"52440bd8-c787-4a9b-99e9-797246119711\") " pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.296672 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.296626    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bknbc\" (UniqueName: \"kubernetes.io/projected/52440bd8-c787-4a9b-99e9-797246119711-kube-api-access-bknbc\") pod \"node-exporter-q8r92\" (UID: \"52440bd8-c787-4a9b-99e9-797246119711\") " pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.323451 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.323403    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:47.358382 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.357879    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q8r92"
Jul 10 23:00:47.373691 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:00:47.373608    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52440bd8_c787_4a9b_99e9_797246119711.slice/crio-e7156b57cf914dd2a2c71d4a9a8e4fb26cc30790918afa39178d7aebee35fe8a WatchSource:0}: Error finding container e7156b57cf914dd2a2c71d4a9a8e4fb26cc30790918afa39178d7aebee35fe8a: Status 404 returned error can't find the container with id e7156b57cf914dd2a2c71d4a9a8e4fb26cc30790918afa39178d7aebee35fe8a
Jul 10 23:00:47.424048 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.424008    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:47.523717 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.523679    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:47.625927 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.625830    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:47.649170 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.649114    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q8r92" event={"ID":"52440bd8-c787-4a9b-99e9-797246119711","Type":"ContainerStarted","Data":"e7156b57cf914dd2a2c71d4a9a8e4fb26cc30790918afa39178d7aebee35fe8a"}
Jul 10 23:00:47.724906 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.724861    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:47.827400 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:47.827337    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:48.021928 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.021886    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:48.095321 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.095287    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:48.101571 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.101538    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 23:00:48.106068 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.106037    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.109037 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.109005    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 10 23:00:48.109522 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.109502    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 10 23:00:48.110001 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.109982    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 10 23:00:48.110236 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.110221    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 10 23:00:48.110513 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.110491    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 10 23:00:48.112071 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.111354    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-kgr2t\""
Jul 10 23:00:48.112071 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.111634    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 10 23:00:48.112071 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.111863    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 10 23:00:48.112282 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.112090    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 10 23:00:48.112332 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.112300    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 10 23:00:48.134390 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.134173    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 23:00:48.195287 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.195242    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/77919191-de58-4fb2-9f9e-a935562d5521-config-out\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.195486 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.195299    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-web-config\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.195486 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.195352    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-config-volume\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.195486 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.195444    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77919191-de58-4fb2-9f9e-a935562d5521-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.195486 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.195476    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw4mh\" (UniqueName: \"kubernetes.io/projected/77919191-de58-4fb2-9f9e-a935562d5521-kube-api-access-fw4mh\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.195653 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.195503    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/77919191-de58-4fb2-9f9e-a935562d5521-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.195653 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.195532    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.195653 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.195557    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.195653 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.195590    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.195653 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.195631    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.195875 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.195657    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77919191-de58-4fb2-9f9e-a935562d5521-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.195875 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.195687    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.195875 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.195738    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/77919191-de58-4fb2-9f9e-a935562d5521-tls-assets\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.236418 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.236354    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:48.296872 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.296782    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77919191-de58-4fb2-9f9e-a935562d5521-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.296872 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.296838    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw4mh\" (UniqueName: \"kubernetes.io/projected/77919191-de58-4fb2-9f9e-a935562d5521-kube-api-access-fw4mh\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.296872 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.296871    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/77919191-de58-4fb2-9f9e-a935562d5521-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.297145 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.296899    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.297264 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.297237    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.297323 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.297297    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.297402 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.297345    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.297402 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.297396    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77919191-de58-4fb2-9f9e-a935562d5521-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.297510 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.297427    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.297510 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.297459    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/77919191-de58-4fb2-9f9e-a935562d5521-tls-assets\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.297608 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.297522    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/77919191-de58-4fb2-9f9e-a935562d5521-config-out\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.297608 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.297550    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-web-config\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.297608 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.297592    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-config-volume\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.297761 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.297659    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/77919191-de58-4fb2-9f9e-a935562d5521-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.297761 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.297728    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77919191-de58-4fb2-9f9e-a935562d5521-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.299002 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:48.298965    2563 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 10 23:00:48.299081 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:00:48.299058    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-secret-alertmanager-main-tls podName:77919191-de58-4fb2-9f9e-a935562d5521 nodeName:}" failed. No retries permitted until 2026-07-10 23:00:48.799035396 +0000 UTC m=+309.685771521 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "77919191-de58-4fb2-9f9e-a935562d5521") : secret "alertmanager-main-tls" not found
Jul 10 23:00:48.300587 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.300562    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.300762 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.300737    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77919191-de58-4fb2-9f9e-a935562d5521-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.301786 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.301757    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-config-volume\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.303474 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.303430    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/77919191-de58-4fb2-9f9e-a935562d5521-tls-assets\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.304188 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.304150    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-web-config\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.305160 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.305119    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/77919191-de58-4fb2-9f9e-a935562d5521-config-out\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.305254 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.305171    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.306494 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.306440    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.307102 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.307078    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.307668 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.307638    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw4mh\" (UniqueName: \"kubernetes.io/projected/77919191-de58-4fb2-9f9e-a935562d5521-kube-api-access-fw4mh\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.337669 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.337626    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:48.604145 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.604056    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:48.802022 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.801992    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:48.805323 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:48.805286    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:49.022698 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:49.022667    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:00:49.661466 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:49.661419    2563 generic.go:356] "Generic (PLEG): container finished" podID="52440bd8-c787-4a9b-99e9-797246119711" containerID="2f2c99c89afd9b87476d2ddfd6d7ad0199933f6689dccc37c690f60d15a6eb41" exitCode=0
Jul 10 23:00:49.661649 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:49.661483    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q8r92" event={"ID":"52440bd8-c787-4a9b-99e9-797246119711","Type":"ContainerDied","Data":"2f2c99c89afd9b87476d2ddfd6d7ad0199933f6689dccc37c690f60d15a6eb41"}
Jul 10 23:00:49.967417 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:49.967305    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:50.035068 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.035030    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-776454d5b4-n8tbg"]
Jul 10 23:00:50.039301 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.039272    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg"
Jul 10 23:00:50.042760 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.042727    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-drd6x\""
Jul 10 23:00:50.042944 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.042824    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 10 23:00:50.043554 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.043535    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-7l70ekkpuskap\""
Jul 10 23:00:50.044233 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.043710    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 10 23:00:50.044233 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.043870    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 10 23:00:50.044233 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.043875    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 10 23:00:50.044233 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.043898    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 10 23:00:50.056645 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.056611    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-776454d5b4-n8tbg"]
Jul 10 23:00:50.069575 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.069535    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:50.144666 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.144630    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:50.215271 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.215226    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/39f19430-c21f-45ee-aae2-b8fec526f77f-secret-grpc-tls\") pod \"thanos-querier-776454d5b4-n8tbg\" (UID: \"39f19430-c21f-45ee-aae2-b8fec526f77f\") " pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg"
Jul 10 23:00:50.215271 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.215270    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc2dk\" (UniqueName: \"kubernetes.io/projected/39f19430-c21f-45ee-aae2-b8fec526f77f-kube-api-access-nc2dk\") pod \"thanos-querier-776454d5b4-n8tbg\" (UID: \"39f19430-c21f-45ee-aae2-b8fec526f77f\") " pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg"
Jul 10 23:00:50.215538 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.215349    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/39f19430-c21f-45ee-aae2-b8fec526f77f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-776454d5b4-n8tbg\" (UID: \"39f19430-c21f-45ee-aae2-b8fec526f77f\") " pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg"
Jul 10 23:00:50.215538 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.215447    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/39f19430-c21f-45ee-aae2-b8fec526f77f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-776454d5b4-n8tbg\" (UID: \"39f19430-c21f-45ee-aae2-b8fec526f77f\") " pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg"
Jul 10 23:00:50.215538 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.215501    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/39f19430-c21f-45ee-aae2-b8fec526f77f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-776454d5b4-n8tbg\" (UID: \"39f19430-c21f-45ee-aae2-b8fec526f77f\") " pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg"
Jul 10 23:00:50.215686 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.215556    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/39f19430-c21f-45ee-aae2-b8fec526f77f-secret-thanos-querier-tls\") pod \"thanos-querier-776454d5b4-n8tbg\" (UID: \"39f19430-c21f-45ee-aae2-b8fec526f77f\") " pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg"
Jul 10 23:00:50.215686 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.215595    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39f19430-c21f-45ee-aae2-b8fec526f77f-metrics-client-ca\") pod \"thanos-querier-776454d5b4-n8tbg\" (UID: \"39f19430-c21f-45ee-aae2-b8fec526f77f\") " pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg"
Jul 10 23:00:50.215686 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.215639    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/39f19430-c21f-45ee-aae2-b8fec526f77f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-776454d5b4-n8tbg\" (UID: \"39f19430-c21f-45ee-aae2-b8fec526f77f\") " pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg"
Jul 10 23:00:50.316634 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.316586    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39f19430-c21f-45ee-aae2-b8fec526f77f-metrics-client-ca\") pod \"thanos-querier-776454d5b4-n8tbg\" (UID: \"39f19430-c21f-45ee-aae2-b8fec526f77f\") " pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg"
Jul 10 23:00:50.316853 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.316658    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/39f19430-c21f-45ee-aae2-b8fec526f77f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-776454d5b4-n8tbg\" (UID: \"39f19430-c21f-45ee-aae2-b8fec526f77f\") " pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg"
Jul 10 23:00:50.316853 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.316711    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/39f19430-c21f-45ee-aae2-b8fec526f77f-secret-grpc-tls\") pod \"thanos-querier-776454d5b4-n8tbg\" (UID: \"39f19430-c21f-45ee-aae2-b8fec526f77f\") " pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg"
Jul 10 23:00:50.316853 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.316739    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc2dk\" (UniqueName: \"kubernetes.io/projected/39f19430-c21f-45ee-aae2-b8fec526f77f-kube-api-access-nc2dk\") pod \"thanos-querier-776454d5b4-n8tbg\" (UID: \"39f19430-c21f-45ee-aae2-b8fec526f77f\") " pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg"
Jul 10 23:00:50.316853 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.316785    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/39f19430-c21f-45ee-aae2-b8fec526f77f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-776454d5b4-n8tbg\" (UID: \"39f19430-c21f-45ee-aae2-b8fec526f77f\") " pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg"
Jul 10 23:00:50.316853 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.316816    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/39f19430-c21f-45ee-aae2-b8fec526f77f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-776454d5b4-n8tbg\" (UID: \"39f19430-c21f-45ee-aae2-b8fec526f77f\") " pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg"
Jul 10 23:00:50.316853 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.316850    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/39f19430-c21f-45ee-aae2-b8fec526f77f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-776454d5b4-n8tbg\" (UID: \"39f19430-c21f-45ee-aae2-b8fec526f77f\") " pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg"
Jul 10 23:00:50.317145 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.316899    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/39f19430-c21f-45ee-aae2-b8fec526f77f-secret-thanos-querier-tls\") pod \"thanos-querier-776454d5b4-n8tbg\" (UID: \"39f19430-c21f-45ee-aae2-b8fec526f77f\") " pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg"
Jul 10 23:00:50.317505 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.317476    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39f19430-c21f-45ee-aae2-b8fec526f77f-metrics-client-ca\") pod \"thanos-querier-776454d5b4-n8tbg\" (UID: \"39f19430-c21f-45ee-aae2-b8fec526f77f\") " pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg"
Jul 10 23:00:50.320012 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.319981    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/39f19430-c21f-45ee-aae2-b8fec526f77f-secret-thanos-querier-tls\") pod \"thanos-querier-776454d5b4-n8tbg\" (UID: \"39f19430-c21f-45ee-aae2-b8fec526f77f\") " pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg"
Jul 10 23:00:50.320254 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.320231    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/39f19430-c21f-45ee-aae2-b8fec526f77f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-776454d5b4-n8tbg\" (UID: \"39f19430-c21f-45ee-aae2-b8fec526f77f\") " pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg"
Jul 10 23:00:50.320649 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.320623    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/39f19430-c21f-45ee-aae2-b8fec526f77f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-776454d5b4-n8tbg\" (UID: \"39f19430-c21f-45ee-aae2-b8fec526f77f\") " pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg"
Jul 10 23:00:50.320741 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.320703    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/39f19430-c21f-45ee-aae2-b8fec526f77f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-776454d5b4-n8tbg\" (UID: \"39f19430-c21f-45ee-aae2-b8fec526f77f\") " pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg"
Jul 10 23:00:50.320884 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.320862    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/39f19430-c21f-45ee-aae2-b8fec526f77f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-776454d5b4-n8tbg\" (UID: \"39f19430-c21f-45ee-aae2-b8fec526f77f\") " pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg"
Jul 10 23:00:50.320930 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.320894    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/39f19430-c21f-45ee-aae2-b8fec526f77f-secret-grpc-tls\") pod \"thanos-querier-776454d5b4-n8tbg\" (UID: \"39f19430-c21f-45ee-aae2-b8fec526f77f\") " pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg"
Jul 10 23:00:50.324907 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.324881    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc2dk\" (UniqueName: \"kubernetes.io/projected/39f19430-c21f-45ee-aae2-b8fec526f77f-kube-api-access-nc2dk\") pod \"thanos-querier-776454d5b4-n8tbg\" (UID: \"39f19430-c21f-45ee-aae2-b8fec526f77f\") " pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg"
Jul 10 23:00:50.351025 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.350973    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg"
Jul 10 23:00:50.570708 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:50.570605    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:51.371138 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:51.371102    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-cb5cb64c4-8fd95"]
Jul 10 23:00:51.376813 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:51.376767    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cb5cb64c4-8fd95"
Jul 10 23:00:51.386446 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:51.386416    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 10 23:00:51.387265 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:51.387214    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cb5cb64c4-8fd95"]
Jul 10 23:00:51.529376 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:51.529325    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/953d6354-ca9c-4dec-843e-ebf7497c4a95-console-oauth-config\") pod \"console-cb5cb64c4-8fd95\" (UID: \"953d6354-ca9c-4dec-843e-ebf7497c4a95\") " pod="openshift-console/console-cb5cb64c4-8fd95"
Jul 10 23:00:51.529571 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:51.529441    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/953d6354-ca9c-4dec-843e-ebf7497c4a95-trusted-ca-bundle\") pod \"console-cb5cb64c4-8fd95\" (UID: \"953d6354-ca9c-4dec-843e-ebf7497c4a95\") " pod="openshift-console/console-cb5cb64c4-8fd95"
Jul 10 23:00:51.529571 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:51.529483    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd4p9\" (UniqueName: \"kubernetes.io/projected/953d6354-ca9c-4dec-843e-ebf7497c4a95-kube-api-access-hd4p9\") pod \"console-cb5cb64c4-8fd95\" (UID: \"953d6354-ca9c-4dec-843e-ebf7497c4a95\") " pod="openshift-console/console-cb5cb64c4-8fd95"
Jul 10 23:00:51.529571 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:51.529521    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/953d6354-ca9c-4dec-843e-ebf7497c4a95-console-serving-cert\") pod \"console-cb5cb64c4-8fd95\" (UID: \"953d6354-ca9c-4dec-843e-ebf7497c4a95\") " pod="openshift-console/console-cb5cb64c4-8fd95"
Jul 10 23:00:51.529707 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:51.529566    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/953d6354-ca9c-4dec-843e-ebf7497c4a95-console-config\") pod \"console-cb5cb64c4-8fd95\" (UID: \"953d6354-ca9c-4dec-843e-ebf7497c4a95\") " pod="openshift-console/console-cb5cb64c4-8fd95"
Jul 10 23:00:51.529707 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:51.529614    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/953d6354-ca9c-4dec-843e-ebf7497c4a95-service-ca\") pod \"console-cb5cb64c4-8fd95\" (UID: \"953d6354-ca9c-4dec-843e-ebf7497c4a95\") " pod="openshift-console/console-cb5cb64c4-8fd95"
Jul 10 23:00:51.529707 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:51.529661    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/953d6354-ca9c-4dec-843e-ebf7497c4a95-oauth-serving-cert\") pod \"console-cb5cb64c4-8fd95\" (UID: \"953d6354-ca9c-4dec-843e-ebf7497c4a95\") " pod="openshift-console/console-cb5cb64c4-8fd95"
Jul 10 23:00:51.575662 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:51.575623    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:51.630890 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:51.630796    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/953d6354-ca9c-4dec-843e-ebf7497c4a95-trusted-ca-bundle\") pod \"console-cb5cb64c4-8fd95\" (UID: \"953d6354-ca9c-4dec-843e-ebf7497c4a95\") " pod="openshift-console/console-cb5cb64c4-8fd95"
Jul 10 23:00:51.630890 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:51.630860    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd4p9\" (UniqueName: \"kubernetes.io/projected/953d6354-ca9c-4dec-843e-ebf7497c4a95-kube-api-access-hd4p9\") pod \"console-cb5cb64c4-8fd95\" (UID: \"953d6354-ca9c-4dec-843e-ebf7497c4a95\") " pod="openshift-console/console-cb5cb64c4-8fd95"
Jul 10 23:00:51.631113 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:51.630898    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/953d6354-ca9c-4dec-843e-ebf7497c4a95-console-serving-cert\") pod \"console-cb5cb64c4-8fd95\" (UID: \"953d6354-ca9c-4dec-843e-ebf7497c4a95\") " pod="openshift-console/console-cb5cb64c4-8fd95"
Jul 10 23:00:51.631113 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:51.631046    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/953d6354-ca9c-4dec-843e-ebf7497c4a95-console-config\") pod \"console-cb5cb64c4-8fd95\" (UID: \"953d6354-ca9c-4dec-843e-ebf7497c4a95\") " pod="openshift-console/console-cb5cb64c4-8fd95"
Jul 10 23:00:51.631113 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:51.631088    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/953d6354-ca9c-4dec-843e-ebf7497c4a95-service-ca\") pod \"console-cb5cb64c4-8fd95\" (UID: \"953d6354-ca9c-4dec-843e-ebf7497c4a95\") " pod="openshift-console/console-cb5cb64c4-8fd95"
Jul 10 23:00:51.631229 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:51.631117    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/953d6354-ca9c-4dec-843e-ebf7497c4a95-oauth-serving-cert\") pod \"console-cb5cb64c4-8fd95\" (UID: \"953d6354-ca9c-4dec-843e-ebf7497c4a95\") " pod="openshift-console/console-cb5cb64c4-8fd95"
Jul 10 23:00:51.631229 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:51.631216    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/953d6354-ca9c-4dec-843e-ebf7497c4a95-console-oauth-config\") pod \"console-cb5cb64c4-8fd95\" (UID: \"953d6354-ca9c-4dec-843e-ebf7497c4a95\") " pod="openshift-console/console-cb5cb64c4-8fd95"
Jul 10 23:00:51.631809 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:51.631758    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/953d6354-ca9c-4dec-843e-ebf7497c4a95-console-config\") pod \"console-cb5cb64c4-8fd95\" (UID: \"953d6354-ca9c-4dec-843e-ebf7497c4a95\") " pod="openshift-console/console-cb5cb64c4-8fd95"
Jul 10 23:00:51.631948 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:51.631871    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/953d6354-ca9c-4dec-843e-ebf7497c4a95-service-ca\") pod \"console-cb5cb64c4-8fd95\" (UID: \"953d6354-ca9c-4dec-843e-ebf7497c4a95\") " pod="openshift-console/console-cb5cb64c4-8fd95"
Jul 10 23:00:51.632184 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:51.632151    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/953d6354-ca9c-4dec-843e-ebf7497c4a95-oauth-serving-cert\") pod \"console-cb5cb64c4-8fd95\" (UID: \"953d6354-ca9c-4dec-843e-ebf7497c4a95\") " pod="openshift-console/console-cb5cb64c4-8fd95"
Jul 10 23:00:51.632873 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:51.632851    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/953d6354-ca9c-4dec-843e-ebf7497c4a95-trusted-ca-bundle\") pod \"console-cb5cb64c4-8fd95\" (UID: \"953d6354-ca9c-4dec-843e-ebf7497c4a95\") " pod="openshift-console/console-cb5cb64c4-8fd95"
Jul 10 23:00:51.634154 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:51.634130    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/953d6354-ca9c-4dec-843e-ebf7497c4a95-console-serving-cert\") pod \"console-cb5cb64c4-8fd95\" (UID: \"953d6354-ca9c-4dec-843e-ebf7497c4a95\") " pod="openshift-console/console-cb5cb64c4-8fd95"
Jul 10 23:00:51.634814 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:51.634792    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/953d6354-ca9c-4dec-843e-ebf7497c4a95-console-oauth-config\") pod \"console-cb5cb64c4-8fd95\" (UID: \"953d6354-ca9c-4dec-843e-ebf7497c4a95\") " pod="openshift-console/console-cb5cb64c4-8fd95"
Jul 10 23:00:51.641110 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:51.641040    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd4p9\" (UniqueName: \"kubernetes.io/projected/953d6354-ca9c-4dec-843e-ebf7497c4a95-kube-api-access-hd4p9\") pod \"console-cb5cb64c4-8fd95\" (UID: \"953d6354-ca9c-4dec-843e-ebf7497c4a95\") " pod="openshift-console/console-cb5cb64c4-8fd95"
Jul 10 23:00:51.690622 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:51.690568    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cb5cb64c4-8fd95"
Jul 10 23:00:51.825542 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:51.825501    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:52.099756 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:52.099715    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77758c754-58r8c"
Jul 10 23:00:52.100006 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:52.099871    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77758c754-58r8c"
Jul 10 23:00:52.105734 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:52.105700    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77758c754-58r8c"
Jul 10 23:00:52.439688 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:52.439598    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:52.538207 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:52.538163    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:52.680094 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:52.680060    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77758c754-58r8c"
Jul 10 23:00:53.255141 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.255101    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 23:00:53.256214 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.256180    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:53.259761 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.259735    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.262299 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.262054    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 10 23:00:53.262299 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.262201    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-snfnn\""
Jul 10 23:00:53.262299 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.262238    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 10 23:00:53.262299 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.262242    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 10 23:00:53.262627 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.262541    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 10 23:00:53.262627 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.262552    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 10 23:00:53.262724 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.262688    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 10 23:00:53.262724 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.262716    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 10 23:00:53.263356 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.263337    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ch12p89tseoff\""
Jul 10 23:00:53.263922 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.263633    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 10 23:00:53.263922 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.263684    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 10 23:00:53.263922 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.263709    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 10 23:00:53.263922 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.263801    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 10 23:00:53.264181 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.264131    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 10 23:00:53.264284 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.264257    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 10 23:00:53.265992 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.265972    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 10 23:00:53.269512 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.269487    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 10 23:00:53.276710 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.276678    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 23:00:53.448262 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.448226    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.448262 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.448267    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.448801 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.448295    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.448801 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.448336    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.448801 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.448430    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.448801 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.448456    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.448801 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.448486    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2bddddf0-512e-466c-9327-b88418a330e4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.448801 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.448504    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-web-config\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.448801 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.448526    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.448801 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.448555    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.448801 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.448632    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.448801 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.448689    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.448801 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.448735    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2bddddf0-512e-466c-9327-b88418a330e4-config-out\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.448801 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.448761    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.448801 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.448789    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.449324 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.448826    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qhrh\" (UniqueName: \"kubernetes.io/projected/2bddddf0-512e-466c-9327-b88418a330e4-kube-api-access-5qhrh\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.449324 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.448888    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.449324 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.448928    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-config\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.449324 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.448956    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.449324 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.448980    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.549866 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.549767    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.549866 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.549817    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.549866 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.549849    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.550149 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.549893    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.550149 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.549938    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2bddddf0-512e-466c-9327-b88418a330e4-config-out\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.550149 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.549962    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.550149 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.549988    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.550149 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.550023    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qhrh\" (UniqueName: \"kubernetes.io/projected/2bddddf0-512e-466c-9327-b88418a330e4-kube-api-access-5qhrh\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.550149 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.550066    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.550149 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.550093    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-config\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.550149 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.550119    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.550149 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.550144    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.550617 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.550185    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.550617 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.550212    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.550617 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.550241    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.550617 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.550264    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.550617 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.550317    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.550617 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.550341    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.550617 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.550396    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2bddddf0-512e-466c-9327-b88418a330e4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.550617 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.550421    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-web-config\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.550617 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.550610    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.551215 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.550871    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.551542 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.551515    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.552217 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.552190    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.554076 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.553672    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.554076 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.553922    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.554076 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.554016    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.554722 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.554504    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.555011 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.554985    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.555077 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.555014    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.556192 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.556091    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.563772 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.557756    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-web-config\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.563772 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.558196    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2bddddf0-512e-466c-9327-b88418a330e4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.563772 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.558294    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2bddddf0-512e-466c-9327-b88418a330e4-config-out\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.563772 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.558416    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.563772 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.558517    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.563772 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.558953    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.563772 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.558978    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.563772 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.560234    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-config\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.566874 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.566845    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qhrh\" (UniqueName: \"kubernetes.io/projected/2bddddf0-512e-466c-9327-b88418a330e4-kube-api-access-5qhrh\") pod \"prometheus-k8s-0\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:53.573539 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:53.573505    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:00:54.411242 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:54.411204    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:54.511934 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:54.511888    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:56.388125 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:56.388038    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cb5cb64c4-8fd95"]
Jul 10 23:00:56.389737 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:00:56.389684    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod953d6354_ca9c_4dec_843e_ebf7497c4a95.slice/crio-06d21d31483794dc688ba49b8cee39112a38a980bb32639772fee76fc964efec WatchSource:0}: Error finding container 06d21d31483794dc688ba49b8cee39112a38a980bb32639772fee76fc964efec: Status 404 returned error can't find the container with id 06d21d31483794dc688ba49b8cee39112a38a980bb32639772fee76fc964efec
Jul 10 23:00:56.402231 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:56.402129    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-776454d5b4-n8tbg"]
Jul 10 23:00:56.636626 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:56.636543    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 23:00:56.639772 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:00:56.639736    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77919191_de58_4fb2_9f9e_a935562d5521.slice/crio-c932cbe3c986b1f950bd1e1bd70c05c2a77602e9b0d5a86171186e43b2c6b959 WatchSource:0}: Error finding container c932cbe3c986b1f950bd1e1bd70c05c2a77602e9b0d5a86171186e43b2c6b959: Status 404 returned error can't find the container with id c932cbe3c986b1f950bd1e1bd70c05c2a77602e9b0d5a86171186e43b2c6b959
Jul 10 23:00:56.641156 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:56.641130    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 23:00:56.645103 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:56.645039    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:56.649160 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:00:56.649135    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bddddf0_512e_466c_9327_b88418a330e4.slice/crio-1df118154b39623f1510b42496c4fe923909a84f37948c073752758630cea40d WatchSource:0}: Error finding container 1df118154b39623f1510b42496c4fe923909a84f37948c073752758630cea40d: Status 404 returned error can't find the container with id 1df118154b39623f1510b42496c4fe923909a84f37948c073752758630cea40d
Jul 10 23:00:56.694834 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:56.694722    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-945ss" event={"ID":"24c76975-a76f-48e0-b1a8-098242d58511","Type":"ContainerStarted","Data":"b0778b8c9c02649b632f60641effce6513ffd5b341bb27d9ef07a2ef10b82f22"}
Jul 10 23:00:56.694991 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:56.694911    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-945ss"
Jul 10 23:00:56.697861 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:56.697826    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q8r92" event={"ID":"52440bd8-c787-4a9b-99e9-797246119711","Type":"ContainerStarted","Data":"a5fe808aee4003b2cbb0a9ef72f86a4b70dd11edc5dbb3623288a13ead326c87"}
Jul 10 23:00:56.698009 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:56.697888    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q8r92" event={"ID":"52440bd8-c787-4a9b-99e9-797246119711","Type":"ContainerStarted","Data":"ee65994f4001c9f620e738f17e1dd8146f543510e143414e9dbce82cc68dcc0e"}
Jul 10 23:00:56.699141 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:56.699111    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg" event={"ID":"39f19430-c21f-45ee-aae2-b8fec526f77f","Type":"ContainerStarted","Data":"e80764ae3aaca05560453cccc9d07d043ec656da22b38659ec55a7edf2f2626d"}
Jul 10 23:00:56.700523 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:56.700495    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77919191-de58-4fb2-9f9e-a935562d5521","Type":"ContainerStarted","Data":"c932cbe3c986b1f950bd1e1bd70c05c2a77602e9b0d5a86171186e43b2c6b959"}
Jul 10 23:00:56.701994 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:56.701963    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bddddf0-512e-466c-9327-b88418a330e4","Type":"ContainerStarted","Data":"1df118154b39623f1510b42496c4fe923909a84f37948c073752758630cea40d"}
Jul 10 23:00:56.703954 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:56.703909    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cb5cb64c4-8fd95" event={"ID":"953d6354-ca9c-4dec-843e-ebf7497c4a95","Type":"ContainerStarted","Data":"737d4524d311b8c75772d443874ab9a01b56d82488a1ea55b1cd67a959c7fb84"}
Jul 10 23:00:56.703954 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:56.703956    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cb5cb64c4-8fd95" event={"ID":"953d6354-ca9c-4dec-843e-ebf7497c4a95","Type":"ContainerStarted","Data":"06d21d31483794dc688ba49b8cee39112a38a980bb32639772fee76fc964efec"}
Jul 10 23:00:56.706350 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:56.706326    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-945ss"
Jul 10 23:00:56.713932 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:56.713880    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-945ss" podStartSLOduration=1.518935162 podStartE2EDuration="18.713864794s" totalImagesPullingTime="17.194929632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:00:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:00:56.712189848 +0000 UTC m=+317.598926001" watchObservedRunningTime="2026-07-10 23:00:56.713864794 +0000 UTC m=+317.600600966"
Jul 10 23:00:56.729210 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:56.729143    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-cb5cb64c4-8fd95" podStartSLOduration=5.729123969 podStartE2EDuration="5.729123969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:00:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:00:56.728014065 +0000 UTC m=+317.614750221" watchObservedRunningTime="2026-07-10 23:00:56.729123969 +0000 UTC m=+317.615860112"
Jul 10 23:00:56.763768 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:56.763670    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-q8r92" podStartSLOduration=8.335332533 podStartE2EDuration="9.763645391s" totalImagesPullingTime="1.339212931s" totalInitContainerRuntime="89.099927ms" isStatefulPod=true podCreationTimestamp="2026-07-10 23:00:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:00:56.761943933 +0000 UTC m=+317.648680097" watchObservedRunningTime="2026-07-10 23:00:56.763645391 +0000 UTC m=+317.650381535"
Jul 10 23:00:57.243851 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:57.243753    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:57.345928 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:57.345820    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:00:59.722430 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:59.722332    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg" event={"ID":"39f19430-c21f-45ee-aae2-b8fec526f77f","Type":"ContainerStarted","Data":"c2755126926de1967fb6b2f3b8bd013932f06f89c8064fa3ddf6311c9757168e"}
Jul 10 23:00:59.722920 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:59.722457    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg" event={"ID":"39f19430-c21f-45ee-aae2-b8fec526f77f","Type":"ContainerStarted","Data":"51e33bb579c75718c7a83c60a78733ccf1154b05a78992846341427f78e72790"}
Jul 10 23:00:59.722920 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:59.722495    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg" event={"ID":"39f19430-c21f-45ee-aae2-b8fec526f77f","Type":"ContainerStarted","Data":"5a581f819dad7e8a44dcf15ba6d8646dc0ff682b438c1b436ef253fca75dfef0"}
Jul 10 23:00:59.724178 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:59.724147    2563 generic.go:356] "Generic (PLEG): container finished" podID="77919191-de58-4fb2-9f9e-a935562d5521" containerID="6854cef15ecdb362e7a20271fe472db56baa82d505e1e345e36b4510a9eca6f6" exitCode=0
Jul 10 23:00:59.724297 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:59.724234    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77919191-de58-4fb2-9f9e-a935562d5521","Type":"ContainerDied","Data":"6854cef15ecdb362e7a20271fe472db56baa82d505e1e345e36b4510a9eca6f6"}
Jul 10 23:00:59.726237 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:59.726136    2563 generic.go:356] "Generic (PLEG): container finished" podID="2bddddf0-512e-466c-9327-b88418a330e4" containerID="e5b73f0955a4c5c82ab1d7fd921e504a83792b82dab39f3a5e10d23420d1ebcd" exitCode=0
Jul 10 23:00:59.726237 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:00:59.726176    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bddddf0-512e-466c-9327-b88418a330e4","Type":"ContainerDied","Data":"e5b73f0955a4c5c82ab1d7fd921e504a83792b82dab39f3a5e10d23420d1ebcd"}
Jul 10 23:01:00.018229 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:00.018191    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cb5cb64c4-8fd95"]
Jul 10 23:01:00.501725 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:00.501678    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-769cc77955-d2747"]
Jul 10 23:01:00.602774 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:00.602730    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:01:00.784626 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:00.784535    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:01:00.885870 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:00.885789    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:01:01.691440 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:01.691402    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-cb5cb64c4-8fd95"
Jul 10 23:01:01.784195 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:01.784081    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:01:01.838141 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:01.838023    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:01:02.753087 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:02.753017    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77919191-de58-4fb2-9f9e-a935562d5521","Type":"ContainerStarted","Data":"8ee2b3bbe415ab1255bd492abbbad04acef8c7e18ebb8ffe90b45d942c465244"}
Jul 10 23:01:02.753274 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:02.753107    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77919191-de58-4fb2-9f9e-a935562d5521","Type":"ContainerStarted","Data":"e2cac26339363173c1329c570509cfee3013d7d44445065ff6f06e4466a1e6c1"}
Jul 10 23:01:02.934282 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:02.934199    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:01:03.034961 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:03.034830    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:01:04.766694 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:04.766638    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bddddf0-512e-466c-9327-b88418a330e4","Type":"ContainerStarted","Data":"bd22df776c0480e0c627cd65c3b42d3b92d7e56284f4baf81d73a877c9375cf2"}
Jul 10 23:01:04.770605 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:04.770264    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg" event={"ID":"39f19430-c21f-45ee-aae2-b8fec526f77f","Type":"ContainerStarted","Data":"875caa4775356b2b3d9232e7c952e80215354eca3253ae27ef9816a8a16a6c2d"}
Jul 10 23:01:04.770733 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:04.770633    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg" event={"ID":"39f19430-c21f-45ee-aae2-b8fec526f77f","Type":"ContainerStarted","Data":"600e3339c6361d8166c7df339d24687f94e8fffac7778a72fa268ff8b2f0f15b"}
Jul 10 23:01:04.776542 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:04.776492    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77919191-de58-4fb2-9f9e-a935562d5521","Type":"ContainerStarted","Data":"922b41c1fd980087ab036120dcc34a630b72070ca4b3a8b48609b9f6cf19b915"}
Jul 10 23:01:04.776670 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:04.776601    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77919191-de58-4fb2-9f9e-a935562d5521","Type":"ContainerStarted","Data":"22111a6372a5eae53c7e979b66e2ff1ec042fe8df049d2812d3de3c659454fad"}
Jul 10 23:01:05.608064 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:05.608029    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:01:05.670670 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:05.670636    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:01:05.785518 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:05.785471    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg" event={"ID":"39f19430-c21f-45ee-aae2-b8fec526f77f","Type":"ContainerStarted","Data":"ac7fa91096c9c1dd86bd417ed88254cd3d263288ba129225428776f4e26c843f"}
Jul 10 23:01:05.785983 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:05.785753    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg"
Jul 10 23:01:05.790513 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:05.790473    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77919191-de58-4fb2-9f9e-a935562d5521","Type":"ContainerStarted","Data":"45fb8e20ea3a031919bca3263ed82a7a27e76b650876ce42bd5852df98b3fd22"}
Jul 10 23:01:05.790662 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:05.790558    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77919191-de58-4fb2-9f9e-a935562d5521","Type":"ContainerStarted","Data":"6e8728fa21520b8a0e04a83d625b12cb312d3e16f28f24078ffc8d27a5516a52"}
Jul 10 23:01:05.792539 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:05.792512    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg"
Jul 10 23:01:05.794863 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:05.794836    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bddddf0-512e-466c-9327-b88418a330e4","Type":"ContainerStarted","Data":"34bc5ff54638f3356f597ccd349797c3127536e38919b5418d8a8b8f9dd37e1f"}
Jul 10 23:01:05.795000 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:05.794894    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bddddf0-512e-466c-9327-b88418a330e4","Type":"ContainerStarted","Data":"c61d4eaf115c0368a8e0b831824ea8cc7dfc7496c939d1e1a9e086f7b99356b3"}
Jul 10 23:01:05.795000 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:05.794923    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bddddf0-512e-466c-9327-b88418a330e4","Type":"ContainerStarted","Data":"2108db9674798ab3098c8f7d67411053da50c56afc713d4d7b04926403a5dc9e"}
Jul 10 23:01:05.795000 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:05.794946    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bddddf0-512e-466c-9327-b88418a330e4","Type":"ContainerStarted","Data":"7a344f68c31b558b8d22905866867ef06d6793dca58c92d6bb8b16df01b410a2"}
Jul 10 23:01:05.795000 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:05.794970    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bddddf0-512e-466c-9327-b88418a330e4","Type":"ContainerStarted","Data":"e583b6b6ab7395735ec3bab03fbfc072be6db48641354ee2e9050dfe5d3476ea"}
Jul 10 23:01:05.812064 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:05.811994    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-776454d5b4-n8tbg" podStartSLOduration=8.060395731 podStartE2EDuration="15.811976679s" totalImagesPullingTime="7.751580948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:00:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:01:05.809253005 +0000 UTC m=+326.695989161" watchObservedRunningTime="2026-07-10 23:01:05.811976679 +0000 UTC m=+326.698712822"
Jul 10 23:01:05.868278 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:05.868165    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=13.003006367 podStartE2EDuration="17.868149745s" totalImagesPullingTime="4.843577511s" totalInitContainerRuntime="21.565867ms" isStatefulPod=false podCreationTimestamp="2026-07-10 23:00:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:01:05.867001626 +0000 UTC m=+326.753737795" watchObservedRunningTime="2026-07-10 23:01:05.868149745 +0000 UTC m=+326.754885888"
Jul 10 23:01:05.900809 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:05.900743    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=5.628197482 podStartE2EDuration="12.900721456s" totalImagesPullingTime="7.248065794s" totalInitContainerRuntime="24.45818ms" isStatefulPod=false podCreationTimestamp="2026-07-10 23:00:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:01:05.897337323 +0000 UTC m=+326.784073499" watchObservedRunningTime="2026-07-10 23:01:05.900721456 +0000 UTC m=+326.787457600"
Jul 10 23:01:08.574113 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:08.574069    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:01:10.604317 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:10.604274    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77758c754-58r8c"]
Jul 10 23:01:10.701804 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:10.701766    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:01:10.820439 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:10.820406    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:01:10.922190 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:10.922099    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:01:11.293689 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:11.293657    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:01:11.384791 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:11.384756    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:01:21.871851 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:21.871815    2563 generic.go:356] "Generic (PLEG): container finished" podID="db2cc68f-5886-4ce1-8807-cee1d5760f99" containerID="067a384822b61f2291e8c802ae61ce68d492e7462eb89e14906d47a9cf8601bb" exitCode=0
Jul 10 23:01:21.872312 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:21.871898    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tf99c" event={"ID":"db2cc68f-5886-4ce1-8807-cee1d5760f99","Type":"ContainerDied","Data":"067a384822b61f2291e8c802ae61ce68d492e7462eb89e14906d47a9cf8601bb"}
Jul 10 23:01:21.872312 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:21.872281    2563 scope.go:117] "RemoveContainer" containerID="067a384822b61f2291e8c802ae61ce68d492e7462eb89e14906d47a9cf8601bb"
Jul 10 23:01:22.878016 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:22.877983    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tf99c" event={"ID":"db2cc68f-5886-4ce1-8807-cee1d5760f99","Type":"ContainerStarted","Data":"c8dc0dcf7b1914cd187f4331827becab88ece8209b8b13754ad5a23e98e5a1e9"}
Jul 10 23:01:25.042842 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.042799    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-cb5cb64c4-8fd95" podUID="953d6354-ca9c-4dec-843e-ebf7497c4a95" containerName="console" containerID="cri-o://737d4524d311b8c75772d443874ab9a01b56d82488a1ea55b1cd67a959c7fb84" gracePeriod=15
Jul 10 23:01:25.329002 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.328967    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cb5cb64c4-8fd95"
Jul 10 23:01:25.381316 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.381275    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hd4p9\" (UniqueName: \"kubernetes.io/projected/953d6354-ca9c-4dec-843e-ebf7497c4a95-kube-api-access-hd4p9\") pod \"953d6354-ca9c-4dec-843e-ebf7497c4a95\" (UID: \"953d6354-ca9c-4dec-843e-ebf7497c4a95\") "
Jul 10 23:01:25.381316 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.381316    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/953d6354-ca9c-4dec-843e-ebf7497c4a95-console-serving-cert\") pod \"953d6354-ca9c-4dec-843e-ebf7497c4a95\" (UID: \"953d6354-ca9c-4dec-843e-ebf7497c4a95\") "
Jul 10 23:01:25.381570 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.381355    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/953d6354-ca9c-4dec-843e-ebf7497c4a95-trusted-ca-bundle\") pod \"953d6354-ca9c-4dec-843e-ebf7497c4a95\" (UID: \"953d6354-ca9c-4dec-843e-ebf7497c4a95\") "
Jul 10 23:01:25.381570 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.381410    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/953d6354-ca9c-4dec-843e-ebf7497c4a95-console-config\") pod \"953d6354-ca9c-4dec-843e-ebf7497c4a95\" (UID: \"953d6354-ca9c-4dec-843e-ebf7497c4a95\") "
Jul 10 23:01:25.381570 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.381446    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/953d6354-ca9c-4dec-843e-ebf7497c4a95-console-oauth-config\") pod \"953d6354-ca9c-4dec-843e-ebf7497c4a95\" (UID: \"953d6354-ca9c-4dec-843e-ebf7497c4a95\") "
Jul 10 23:01:25.381570 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.381468    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/953d6354-ca9c-4dec-843e-ebf7497c4a95-service-ca\") pod \"953d6354-ca9c-4dec-843e-ebf7497c4a95\" (UID: \"953d6354-ca9c-4dec-843e-ebf7497c4a95\") "
Jul 10 23:01:25.381884 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.381854    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/953d6354-ca9c-4dec-843e-ebf7497c4a95-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "953d6354-ca9c-4dec-843e-ebf7497c4a95" (UID: "953d6354-ca9c-4dec-843e-ebf7497c4a95"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:01:25.381884 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.381871    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/953d6354-ca9c-4dec-843e-ebf7497c4a95-console-config" (OuterVolumeSpecName: "console-config") pod "953d6354-ca9c-4dec-843e-ebf7497c4a95" (UID: "953d6354-ca9c-4dec-843e-ebf7497c4a95"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:01:25.382018 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.381859    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/953d6354-ca9c-4dec-843e-ebf7497c4a95-oauth-serving-cert\") pod \"953d6354-ca9c-4dec-843e-ebf7497c4a95\" (UID: \"953d6354-ca9c-4dec-843e-ebf7497c4a95\") "
Jul 10 23:01:25.382018 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.381865    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/953d6354-ca9c-4dec-843e-ebf7497c4a95-service-ca" (OuterVolumeSpecName: "service-ca") pod "953d6354-ca9c-4dec-843e-ebf7497c4a95" (UID: "953d6354-ca9c-4dec-843e-ebf7497c4a95"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:01:25.382545 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.382137    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/953d6354-ca9c-4dec-843e-ebf7497c4a95-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "953d6354-ca9c-4dec-843e-ebf7497c4a95" (UID: "953d6354-ca9c-4dec-843e-ebf7497c4a95"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:01:25.382545 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.382292    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/953d6354-ca9c-4dec-843e-ebf7497c4a95-trusted-ca-bundle\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:01:25.382545 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.382312    2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/953d6354-ca9c-4dec-843e-ebf7497c4a95-console-config\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:01:25.382545 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.382326    2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/953d6354-ca9c-4dec-843e-ebf7497c4a95-service-ca\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:01:25.382545 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.382378    2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/953d6354-ca9c-4dec-843e-ebf7497c4a95-oauth-serving-cert\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:01:25.383851 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.383824    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/953d6354-ca9c-4dec-843e-ebf7497c4a95-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "953d6354-ca9c-4dec-843e-ebf7497c4a95" (UID: "953d6354-ca9c-4dec-843e-ebf7497c4a95"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:01:25.384067 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.384040    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/953d6354-ca9c-4dec-843e-ebf7497c4a95-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "953d6354-ca9c-4dec-843e-ebf7497c4a95" (UID: "953d6354-ca9c-4dec-843e-ebf7497c4a95"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:01:25.384178 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.384101    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/953d6354-ca9c-4dec-843e-ebf7497c4a95-kube-api-access-hd4p9" (OuterVolumeSpecName: "kube-api-access-hd4p9") pod "953d6354-ca9c-4dec-843e-ebf7497c4a95" (UID: "953d6354-ca9c-4dec-843e-ebf7497c4a95"). InnerVolumeSpecName "kube-api-access-hd4p9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:01:25.483253 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.483215    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hd4p9\" (UniqueName: \"kubernetes.io/projected/953d6354-ca9c-4dec-843e-ebf7497c4a95-kube-api-access-hd4p9\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:01:25.483253 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.483242    2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/953d6354-ca9c-4dec-843e-ebf7497c4a95-console-serving-cert\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:01:25.483253 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.483251    2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/953d6354-ca9c-4dec-843e-ebf7497c4a95-console-oauth-config\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:01:25.542577 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.542538    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-769cc77955-d2747" podUID="e7f0af82-786b-4b48-be84-9ced1047ff52" containerName="registry" containerID="cri-o://c995a3e52edb0293b3a881b7a5d8379fb9f58495ffe24be02697e297634b1a86" gracePeriod=30
Jul 10 23:01:25.821159 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.821080    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 23:01:25.886911 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.886871    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e7f0af82-786b-4b48-be84-9ced1047ff52-trusted-ca\") pod \"e7f0af82-786b-4b48-be84-9ced1047ff52\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") "
Jul 10 23:01:25.886911 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.886911    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftm6k\" (UniqueName: \"kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-kube-api-access-ftm6k\") pod \"e7f0af82-786b-4b48-be84-9ced1047ff52\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") "
Jul 10 23:01:25.887145 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.886938    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-bound-sa-token\") pod \"e7f0af82-786b-4b48-be84-9ced1047ff52\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") "
Jul 10 23:01:25.887145 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.887042    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e7f0af82-786b-4b48-be84-9ced1047ff52-image-registry-private-configuration\") pod \"e7f0af82-786b-4b48-be84-9ced1047ff52\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") "
Jul 10 23:01:25.887145 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.887102    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls\") pod \"e7f0af82-786b-4b48-be84-9ced1047ff52\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") "
Jul 10 23:01:25.887250 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.887171    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-certificates\") pod \"e7f0af82-786b-4b48-be84-9ced1047ff52\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") "
Jul 10 23:01:25.887250 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.887198    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e7f0af82-786b-4b48-be84-9ced1047ff52-ca-trust-extracted\") pod \"e7f0af82-786b-4b48-be84-9ced1047ff52\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") "
Jul 10 23:01:25.887250 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.887234    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e7f0af82-786b-4b48-be84-9ced1047ff52-installation-pull-secrets\") pod \"e7f0af82-786b-4b48-be84-9ced1047ff52\" (UID: \"e7f0af82-786b-4b48-be84-9ced1047ff52\") "
Jul 10 23:01:25.887627 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.887445    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7f0af82-786b-4b48-be84-9ced1047ff52-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e7f0af82-786b-4b48-be84-9ced1047ff52" (UID: "e7f0af82-786b-4b48-be84-9ced1047ff52"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:01:25.889539 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.888148    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e7f0af82-786b-4b48-be84-9ced1047ff52" (UID: "e7f0af82-786b-4b48-be84-9ced1047ff52"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:01:25.890176 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.889934    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-kube-api-access-ftm6k" (OuterVolumeSpecName: "kube-api-access-ftm6k") pod "e7f0af82-786b-4b48-be84-9ced1047ff52" (UID: "e7f0af82-786b-4b48-be84-9ced1047ff52"). InnerVolumeSpecName "kube-api-access-ftm6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:01:25.890176 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.890006    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e7f0af82-786b-4b48-be84-9ced1047ff52" (UID: "e7f0af82-786b-4b48-be84-9ced1047ff52"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:01:25.891034 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.891003    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7f0af82-786b-4b48-be84-9ced1047ff52-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e7f0af82-786b-4b48-be84-9ced1047ff52" (UID: "e7f0af82-786b-4b48-be84-9ced1047ff52"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:01:25.895841 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.891289    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e7f0af82-786b-4b48-be84-9ced1047ff52" (UID: "e7f0af82-786b-4b48-be84-9ced1047ff52"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:01:25.895841 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.891562    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7f0af82-786b-4b48-be84-9ced1047ff52-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e7f0af82-786b-4b48-be84-9ced1047ff52" (UID: "e7f0af82-786b-4b48-be84-9ced1047ff52"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:01:25.898760 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.898732    2563 generic.go:356] "Generic (PLEG): container finished" podID="e7f0af82-786b-4b48-be84-9ced1047ff52" containerID="c995a3e52edb0293b3a881b7a5d8379fb9f58495ffe24be02697e297634b1a86" exitCode=0
Jul 10 23:01:25.898888 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.898790    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-769cc77955-d2747"
Jul 10 23:01:25.898888 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.898820    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-769cc77955-d2747" event={"ID":"e7f0af82-786b-4b48-be84-9ced1047ff52","Type":"ContainerDied","Data":"c995a3e52edb0293b3a881b7a5d8379fb9f58495ffe24be02697e297634b1a86"}
Jul 10 23:01:25.898995 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.898906    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-769cc77955-d2747" event={"ID":"e7f0af82-786b-4b48-be84-9ced1047ff52","Type":"ContainerDied","Data":"668d108ae8cbe4975c8bfb44d736090536b1d7fcc69b616bda079a7e630ca03b"}
Jul 10 23:01:25.898995 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.898937    2563 scope.go:117] "RemoveContainer" containerID="c995a3e52edb0293b3a881b7a5d8379fb9f58495ffe24be02697e297634b1a86"
Jul 10 23:01:25.899674 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.899648    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7f0af82-786b-4b48-be84-9ced1047ff52-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e7f0af82-786b-4b48-be84-9ced1047ff52" (UID: "e7f0af82-786b-4b48-be84-9ced1047ff52"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:01:25.900232 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.900215    2563 generic.go:356] "Generic (PLEG): container finished" podID="953d6354-ca9c-4dec-843e-ebf7497c4a95" containerID="737d4524d311b8c75772d443874ab9a01b56d82488a1ea55b1cd67a959c7fb84" exitCode=0
Jul 10 23:01:25.900296 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.900244    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cb5cb64c4-8fd95" event={"ID":"953d6354-ca9c-4dec-843e-ebf7497c4a95","Type":"ContainerDied","Data":"737d4524d311b8c75772d443874ab9a01b56d82488a1ea55b1cd67a959c7fb84"}
Jul 10 23:01:25.900296 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.900273    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cb5cb64c4-8fd95" event={"ID":"953d6354-ca9c-4dec-843e-ebf7497c4a95","Type":"ContainerDied","Data":"06d21d31483794dc688ba49b8cee39112a38a980bb32639772fee76fc964efec"}
Jul 10 23:01:25.900445 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.900290    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cb5cb64c4-8fd95"
Jul 10 23:01:25.917231 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.917193    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cb5cb64c4-8fd95"]
Jul 10 23:01:25.918719 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.918695    2563 scope.go:117] "RemoveContainer" containerID="c995a3e52edb0293b3a881b7a5d8379fb9f58495ffe24be02697e297634b1a86"
Jul 10 23:01:25.919055 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:01:25.919019    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c995a3e52edb0293b3a881b7a5d8379fb9f58495ffe24be02697e297634b1a86\": container with ID starting with c995a3e52edb0293b3a881b7a5d8379fb9f58495ffe24be02697e297634b1a86 not found: ID does not exist" containerID="c995a3e52edb0293b3a881b7a5d8379fb9f58495ffe24be02697e297634b1a86"
Jul 10 23:01:25.919174 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.919064    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c995a3e52edb0293b3a881b7a5d8379fb9f58495ffe24be02697e297634b1a86"} err="failed to get container status \"c995a3e52edb0293b3a881b7a5d8379fb9f58495ffe24be02697e297634b1a86\": rpc error: code = NotFound desc = could not find container \"c995a3e52edb0293b3a881b7a5d8379fb9f58495ffe24be02697e297634b1a86\": container with ID starting with c995a3e52edb0293b3a881b7a5d8379fb9f58495ffe24be02697e297634b1a86 not found: ID does not exist"
Jul 10 23:01:25.919174 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.919107    2563 scope.go:117] "RemoveContainer" containerID="737d4524d311b8c75772d443874ab9a01b56d82488a1ea55b1cd67a959c7fb84"
Jul 10 23:01:25.921168 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.921151    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-cb5cb64c4-8fd95"]
Jul 10 23:01:25.927815 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.927794    2563 scope.go:117] "RemoveContainer" containerID="737d4524d311b8c75772d443874ab9a01b56d82488a1ea55b1cd67a959c7fb84"
Jul 10 23:01:25.928165 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:01:25.928136    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"737d4524d311b8c75772d443874ab9a01b56d82488a1ea55b1cd67a959c7fb84\": container with ID starting with 737d4524d311b8c75772d443874ab9a01b56d82488a1ea55b1cd67a959c7fb84 not found: ID does not exist" containerID="737d4524d311b8c75772d443874ab9a01b56d82488a1ea55b1cd67a959c7fb84"
Jul 10 23:01:25.928269 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.928166    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"737d4524d311b8c75772d443874ab9a01b56d82488a1ea55b1cd67a959c7fb84"} err="failed to get container status \"737d4524d311b8c75772d443874ab9a01b56d82488a1ea55b1cd67a959c7fb84\": rpc error: code = NotFound desc = could not find container \"737d4524d311b8c75772d443874ab9a01b56d82488a1ea55b1cd67a959c7fb84\": container with ID starting with 737d4524d311b8c75772d443874ab9a01b56d82488a1ea55b1cd67a959c7fb84 not found: ID does not exist"
Jul 10 23:01:25.989486 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.989446    2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e7f0af82-786b-4b48-be84-9ced1047ff52-image-registry-private-configuration\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:01:25.989486 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.989477    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-tls\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:01:25.989486 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.989487    2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e7f0af82-786b-4b48-be84-9ced1047ff52-registry-certificates\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:01:25.989486 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.989496    2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e7f0af82-786b-4b48-be84-9ced1047ff52-ca-trust-extracted\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:01:25.989747 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.989505    2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e7f0af82-786b-4b48-be84-9ced1047ff52-installation-pull-secrets\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:01:25.989747 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.989514    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e7f0af82-786b-4b48-be84-9ced1047ff52-trusted-ca\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:01:25.989747 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.989521    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftm6k\" (UniqueName: \"kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-kube-api-access-ftm6k\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:01:25.989747 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:25.989530    2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e7f0af82-786b-4b48-be84-9ced1047ff52-bound-sa-token\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:01:26.222577 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:26.222541    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-769cc77955-d2747"]
Jul 10 23:01:26.226142 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:26.226109    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-769cc77955-d2747"]
Jul 10 23:01:26.237862 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:26.237826    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:01:26.292405 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:26.292373    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:01:27.555542 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:27.555500    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="953d6354-ca9c-4dec-843e-ebf7497c4a95" path="/var/lib/kubelet/pods/953d6354-ca9c-4dec-843e-ebf7497c4a95/volumes"
Jul 10 23:01:27.555930 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:27.555915    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7f0af82-786b-4b48-be84-9ced1047ff52" path="/var/lib/kubelet/pods/e7f0af82-786b-4b48-be84-9ced1047ff52/volumes"
Jul 10 23:01:33.170820 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:33.170733    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:01:33.272127 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:33.272090    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:01:33.372789 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:33.372755    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:01:35.633067 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:35.633004    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-77758c754-58r8c" podUID="55ee7f77-d691-4e70-a9fc-35cb36f9c56d" containerName="console" containerID="cri-o://86a45fdbb16e17fb819e77d198a97cc4edb791611de089262cb4fa8aa9b38a16" gracePeriod=15
Jul 10 23:01:35.925488 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:35.925447    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77758c754-58r8c"
Jul 10 23:01:35.956183 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:35.956141    2563 generic.go:356] "Generic (PLEG): container finished" podID="55ee7f77-d691-4e70-a9fc-35cb36f9c56d" containerID="86a45fdbb16e17fb819e77d198a97cc4edb791611de089262cb4fa8aa9b38a16" exitCode=0
Jul 10 23:01:35.956183 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:35.956178    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77758c754-58r8c" event={"ID":"55ee7f77-d691-4e70-a9fc-35cb36f9c56d","Type":"ContainerDied","Data":"86a45fdbb16e17fb819e77d198a97cc4edb791611de089262cb4fa8aa9b38a16"}
Jul 10 23:01:35.956471 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:35.956235    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77758c754-58r8c" event={"ID":"55ee7f77-d691-4e70-a9fc-35cb36f9c56d","Type":"ContainerDied","Data":"a09c967b7efd727c7bf635347608ee4af73d631dd1e82d1b142fbcbdda6a84df"}
Jul 10 23:01:35.956471 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:35.956259    2563 scope.go:117] "RemoveContainer" containerID="86a45fdbb16e17fb819e77d198a97cc4edb791611de089262cb4fa8aa9b38a16"
Jul 10 23:01:35.956471 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:35.956184    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77758c754-58r8c"
Jul 10 23:01:35.966917 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:35.966896    2563 scope.go:117] "RemoveContainer" containerID="86a45fdbb16e17fb819e77d198a97cc4edb791611de089262cb4fa8aa9b38a16"
Jul 10 23:01:35.967270 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:01:35.967251    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86a45fdbb16e17fb819e77d198a97cc4edb791611de089262cb4fa8aa9b38a16\": container with ID starting with 86a45fdbb16e17fb819e77d198a97cc4edb791611de089262cb4fa8aa9b38a16 not found: ID does not exist" containerID="86a45fdbb16e17fb819e77d198a97cc4edb791611de089262cb4fa8aa9b38a16"
Jul 10 23:01:35.967336 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:35.967281    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86a45fdbb16e17fb819e77d198a97cc4edb791611de089262cb4fa8aa9b38a16"} err="failed to get container status \"86a45fdbb16e17fb819e77d198a97cc4edb791611de089262cb4fa8aa9b38a16\": rpc error: code = NotFound desc = could not find container \"86a45fdbb16e17fb819e77d198a97cc4edb791611de089262cb4fa8aa9b38a16\": container with ID starting with 86a45fdbb16e17fb819e77d198a97cc4edb791611de089262cb4fa8aa9b38a16 not found: ID does not exist"
Jul 10 23:01:35.977574 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:35.977535    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-console-oauth-config\") pod \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\" (UID: \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\") "
Jul 10 23:01:35.977726 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:35.977586    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dn79m\" (UniqueName: \"kubernetes.io/projected/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-kube-api-access-dn79m\") pod \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\" (UID: \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\") "
Jul 10 23:01:35.977726 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:35.977616    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-service-ca\") pod \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\" (UID: \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\") "
Jul 10 23:01:35.977726 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:35.977665    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-console-serving-cert\") pod \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\" (UID: \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\") "
Jul 10 23:01:35.977726 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:35.977724    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-console-config\") pod \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\" (UID: \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\") "
Jul 10 23:01:35.977891 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:35.977744    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-oauth-serving-cert\") pod \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\" (UID: \"55ee7f77-d691-4e70-a9fc-35cb36f9c56d\") "
Jul 10 23:01:35.978148 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:35.978118    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-console-config" (OuterVolumeSpecName: "console-config") pod "55ee7f77-d691-4e70-a9fc-35cb36f9c56d" (UID: "55ee7f77-d691-4e70-a9fc-35cb36f9c56d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:01:35.978148 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:35.978130    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-service-ca" (OuterVolumeSpecName: "service-ca") pod "55ee7f77-d691-4e70-a9fc-35cb36f9c56d" (UID: "55ee7f77-d691-4e70-a9fc-35cb36f9c56d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:01:35.978278 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:35.978187    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "55ee7f77-d691-4e70-a9fc-35cb36f9c56d" (UID: "55ee7f77-d691-4e70-a9fc-35cb36f9c56d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:01:35.979984 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:35.979953    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "55ee7f77-d691-4e70-a9fc-35cb36f9c56d" (UID: "55ee7f77-d691-4e70-a9fc-35cb36f9c56d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:01:35.979984 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:35.979969    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "55ee7f77-d691-4e70-a9fc-35cb36f9c56d" (UID: "55ee7f77-d691-4e70-a9fc-35cb36f9c56d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:01:35.980111 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:35.979991    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-kube-api-access-dn79m" (OuterVolumeSpecName: "kube-api-access-dn79m") pod "55ee7f77-d691-4e70-a9fc-35cb36f9c56d" (UID: "55ee7f77-d691-4e70-a9fc-35cb36f9c56d"). InnerVolumeSpecName "kube-api-access-dn79m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:01:36.079176 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:36.079129    2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-console-serving-cert\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:01:36.079176 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:36.079165    2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-console-config\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:01:36.079176 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:36.079176    2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-oauth-serving-cert\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:01:36.079176 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:36.079185    2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-console-oauth-config\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:01:36.079486 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:36.079195    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dn79m\" (UniqueName: \"kubernetes.io/projected/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-kube-api-access-dn79m\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:01:36.079486 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:36.079204    2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/55ee7f77-d691-4e70-a9fc-35cb36f9c56d-service-ca\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:01:36.277145 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:36.277106    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77758c754-58r8c"]
Jul 10 23:01:36.280559 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:36.280526    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77758c754-58r8c"]
Jul 10 23:01:36.962754 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:36.962661    2563 generic.go:356] "Generic (PLEG): container finished" podID="5dd8a036-d481-4d56-9e10-1dc8e52983ac" containerID="92b33daf1228843c502f0e16df0a84e1b0312063fd34eb2f2de853646a83d10b" exitCode=0
Jul 10 23:01:36.962754 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:36.962737    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bxm84" event={"ID":"5dd8a036-d481-4d56-9e10-1dc8e52983ac","Type":"ContainerDied","Data":"92b33daf1228843c502f0e16df0a84e1b0312063fd34eb2f2de853646a83d10b"}
Jul 10 23:01:36.963214 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:36.963074    2563 scope.go:117] "RemoveContainer" containerID="92b33daf1228843c502f0e16df0a84e1b0312063fd34eb2f2de853646a83d10b"
Jul 10 23:01:37.558563 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:37.556990    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55ee7f77-d691-4e70-a9fc-35cb36f9c56d" path="/var/lib/kubelet/pods/55ee7f77-d691-4e70-a9fc-35cb36f9c56d/volumes"
Jul 10 23:01:37.970669 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:37.970567    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-bxm84" event={"ID":"5dd8a036-d481-4d56-9e10-1dc8e52983ac","Type":"ContainerStarted","Data":"0e67078107457a9dde094848e93fbb87b96f9b0c5dbb8e1833bf97e93168978f"}
Jul 10 23:01:41.989057 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:41.989011    2563 generic.go:356] "Generic (PLEG): container finished" podID="24725b9a-723c-41d5-959d-4ae184f4d456" containerID="d500df93f620bbded7576f62dadd04dedaf4d6101d25ff7c73ff9e798989cade" exitCode=0
Jul 10 23:01:41.989606 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:41.989080    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-hwd7d" event={"ID":"24725b9a-723c-41d5-959d-4ae184f4d456","Type":"ContainerDied","Data":"d500df93f620bbded7576f62dadd04dedaf4d6101d25ff7c73ff9e798989cade"}
Jul 10 23:01:41.989675 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:41.989619    2563 scope.go:117] "RemoveContainer" containerID="d500df93f620bbded7576f62dadd04dedaf4d6101d25ff7c73ff9e798989cade"
Jul 10 23:01:42.999021 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:42.998982    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-hwd7d" event={"ID":"24725b9a-723c-41d5-959d-4ae184f4d456","Type":"ContainerStarted","Data":"ee5f61d96024da097e86c09a866f0e1da3516c8ccda322ef2925f06555cabeba"}
Jul 10 23:01:52.525828 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:52.525788    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs\") pod \"network-metrics-daemon-w9kxw\" (UID: \"9829a27f-3dbd-44c7-b278-f26beb69f82a\") " pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 23:01:52.528419 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:52.528388    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9829a27f-3dbd-44c7-b278-f26beb69f82a-metrics-certs\") pod \"network-metrics-daemon-w9kxw\" (UID: \"9829a27f-3dbd-44c7-b278-f26beb69f82a\") " pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 23:01:52.554120 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:52.554093    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jr855\""
Jul 10 23:01:52.562038 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:52.562012    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w9kxw"
Jul 10 23:01:52.701451 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:52.701424    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w9kxw"]
Jul 10 23:01:52.704052 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:01:52.704026    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9829a27f_3dbd_44c7_b278_f26beb69f82a.slice/crio-0eb9d4381509f3f6195907de3250358a9f40a6752eaddf1c5d2c62b866028f20 WatchSource:0}: Error finding container 0eb9d4381509f3f6195907de3250358a9f40a6752eaddf1c5d2c62b866028f20: Status 404 returned error can't find the container with id 0eb9d4381509f3f6195907de3250358a9f40a6752eaddf1c5d2c62b866028f20
Jul 10 23:01:53.041564 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:53.041524    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w9kxw" event={"ID":"9829a27f-3dbd-44c7-b278-f26beb69f82a","Type":"ContainerStarted","Data":"0eb9d4381509f3f6195907de3250358a9f40a6752eaddf1c5d2c62b866028f20"}
Jul 10 23:01:53.574490 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:53.574441    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:01:53.596419 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:53.596386    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:01:54.047816 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:54.047784    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w9kxw" event={"ID":"9829a27f-3dbd-44c7-b278-f26beb69f82a","Type":"ContainerStarted","Data":"d3ba196f98ce520f32212f410170787279f3b1993cad537b3205a548fac5024b"}
Jul 10 23:01:54.048012 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:54.047837    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w9kxw" event={"ID":"9829a27f-3dbd-44c7-b278-f26beb69f82a","Type":"ContainerStarted","Data":"e841d82de635bdb4271939d92c0634a16c7ddf9b866bdbbf91d3cb24d86b6b71"}
Jul 10 23:01:54.065716 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:54.065688    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:01:54.095435 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:54.095296    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-w9kxw" podStartSLOduration=374.118056998 podStartE2EDuration="6m15.095279089s" totalImagesPullingTime="977.222091ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 22:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:01:54.061798451 +0000 UTC m=+374.948534595" watchObservedRunningTime="2026-07-10 23:01:54.095279089 +0000 UTC m=+374.982015257"
Jul 10 23:01:54.220745 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:54.220704    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:01:54.269856 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:01:54.269820    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:02:01.364991 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:01.364949    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:02:01.418733 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:01.418701    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:02:01.483522 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:01.483481    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:02:03.110107 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:03.110062    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:02:03.162381 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:03.162322    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:02:07.342922 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:07.342891    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 23:02:07.343344 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:07.343205    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="77919191-de58-4fb2-9f9e-a935562d5521" containerName="alertmanager" containerID="cri-o://e2cac26339363173c1329c570509cfee3013d7d44445065ff6f06e4466a1e6c1" gracePeriod=120
Jul 10 23:02:07.343344 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:07.343271    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="77919191-de58-4fb2-9f9e-a935562d5521" containerName="kube-rbac-proxy-metric" containerID="cri-o://6e8728fa21520b8a0e04a83d625b12cb312d3e16f28f24078ffc8d27a5516a52" gracePeriod=120
Jul 10 23:02:07.343501 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:07.343310    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="77919191-de58-4fb2-9f9e-a935562d5521" containerName="kube-rbac-proxy-web" containerID="cri-o://22111a6372a5eae53c7e979b66e2ff1ec042fe8df049d2812d3de3c659454fad" gracePeriod=120
Jul 10 23:02:07.343501 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:07.343335    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="77919191-de58-4fb2-9f9e-a935562d5521" containerName="prom-label-proxy" containerID="cri-o://45fb8e20ea3a031919bca3263ed82a7a27e76b650876ce42bd5852df98b3fd22" gracePeriod=120
Jul 10 23:02:07.343501 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:07.343383    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="77919191-de58-4fb2-9f9e-a935562d5521" containerName="config-reloader" containerID="cri-o://8ee2b3bbe415ab1255bd492abbbad04acef8c7e18ebb8ffe90b45d942c465244" gracePeriod=120
Jul 10 23:02:07.343501 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:07.343392    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="77919191-de58-4fb2-9f9e-a935562d5521" containerName="kube-rbac-proxy" containerID="cri-o://922b41c1fd980087ab036120dcc34a630b72070ca4b3a8b48609b9f6cf19b915" gracePeriod=120
Jul 10 23:02:07.684738 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:07.684654    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:02:07.784387 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:07.784325    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:02:07.882220 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:07.882187    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:02:08.117993 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.117962    2563 generic.go:356] "Generic (PLEG): container finished" podID="77919191-de58-4fb2-9f9e-a935562d5521" containerID="45fb8e20ea3a031919bca3263ed82a7a27e76b650876ce42bd5852df98b3fd22" exitCode=0
Jul 10 23:02:08.117993 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.117986    2563 generic.go:356] "Generic (PLEG): container finished" podID="77919191-de58-4fb2-9f9e-a935562d5521" containerID="922b41c1fd980087ab036120dcc34a630b72070ca4b3a8b48609b9f6cf19b915" exitCode=0
Jul 10 23:02:08.117993 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.117994    2563 generic.go:356] "Generic (PLEG): container finished" podID="77919191-de58-4fb2-9f9e-a935562d5521" containerID="8ee2b3bbe415ab1255bd492abbbad04acef8c7e18ebb8ffe90b45d942c465244" exitCode=0
Jul 10 23:02:08.117993 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.117999    2563 generic.go:356] "Generic (PLEG): container finished" podID="77919191-de58-4fb2-9f9e-a935562d5521" containerID="e2cac26339363173c1329c570509cfee3013d7d44445065ff6f06e4466a1e6c1" exitCode=0
Jul 10 23:02:08.118298 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.118041    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77919191-de58-4fb2-9f9e-a935562d5521","Type":"ContainerDied","Data":"45fb8e20ea3a031919bca3263ed82a7a27e76b650876ce42bd5852df98b3fd22"}
Jul 10 23:02:08.118298 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.118107    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77919191-de58-4fb2-9f9e-a935562d5521","Type":"ContainerDied","Data":"922b41c1fd980087ab036120dcc34a630b72070ca4b3a8b48609b9f6cf19b915"}
Jul 10 23:02:08.118298 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.118135    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77919191-de58-4fb2-9f9e-a935562d5521","Type":"ContainerDied","Data":"8ee2b3bbe415ab1255bd492abbbad04acef8c7e18ebb8ffe90b45d942c465244"}
Jul 10 23:02:08.118298 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.118160    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77919191-de58-4fb2-9f9e-a935562d5521","Type":"ContainerDied","Data":"e2cac26339363173c1329c570509cfee3013d7d44445065ff6f06e4466a1e6c1"}
Jul 10 23:02:08.412597 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.412566    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:02:08.474428 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.474347    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:02:08.614808 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.614779    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:08.675520 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.675438    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-secret-alertmanager-main-tls\") pod \"77919191-de58-4fb2-9f9e-a935562d5521\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") "
Jul 10 23:02:08.675520 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.675485    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fw4mh\" (UniqueName: \"kubernetes.io/projected/77919191-de58-4fb2-9f9e-a935562d5521-kube-api-access-fw4mh\") pod \"77919191-de58-4fb2-9f9e-a935562d5521\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") "
Jul 10 23:02:08.675520 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.675510    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/77919191-de58-4fb2-9f9e-a935562d5521-tls-assets\") pod \"77919191-de58-4fb2-9f9e-a935562d5521\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") "
Jul 10 23:02:08.675785 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.675528    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-secret-alertmanager-kube-rbac-proxy\") pod \"77919191-de58-4fb2-9f9e-a935562d5521\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") "
Jul 10 23:02:08.675785 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.675549    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77919191-de58-4fb2-9f9e-a935562d5521-alertmanager-trusted-ca-bundle\") pod \"77919191-de58-4fb2-9f9e-a935562d5521\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") "
Jul 10 23:02:08.675785 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.675569    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-config-volume\") pod \"77919191-de58-4fb2-9f9e-a935562d5521\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") "
Jul 10 23:02:08.675785 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.675590    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-cluster-tls-config\") pod \"77919191-de58-4fb2-9f9e-a935562d5521\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") "
Jul 10 23:02:08.676025 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.675986    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-web-config\") pod \"77919191-de58-4fb2-9f9e-a935562d5521\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") "
Jul 10 23:02:08.676083 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.676016    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77919191-de58-4fb2-9f9e-a935562d5521-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "77919191-de58-4fb2-9f9e-a935562d5521" (UID: "77919191-de58-4fb2-9f9e-a935562d5521"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:02:08.676083 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.676038    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/77919191-de58-4fb2-9f9e-a935562d5521-alertmanager-main-db\") pod \"77919191-de58-4fb2-9f9e-a935562d5521\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") "
Jul 10 23:02:08.676194 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.676085    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77919191-de58-4fb2-9f9e-a935562d5521-metrics-client-ca\") pod \"77919191-de58-4fb2-9f9e-a935562d5521\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") "
Jul 10 23:02:08.676312 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.676215    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-secret-alertmanager-kube-rbac-proxy-metric\") pod \"77919191-de58-4fb2-9f9e-a935562d5521\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") "
Jul 10 23:02:08.676400 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.676328    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-secret-alertmanager-kube-rbac-proxy-web\") pod \"77919191-de58-4fb2-9f9e-a935562d5521\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") "
Jul 10 23:02:08.676662 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.676600    2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77919191-de58-4fb2-9f9e-a935562d5521-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:08.677572 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.677542    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77919191-de58-4fb2-9f9e-a935562d5521-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "77919191-de58-4fb2-9f9e-a935562d5521" (UID: "77919191-de58-4fb2-9f9e-a935562d5521"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:02:08.677732 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.677710    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77919191-de58-4fb2-9f9e-a935562d5521-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "77919191-de58-4fb2-9f9e-a935562d5521" (UID: "77919191-de58-4fb2-9f9e-a935562d5521"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:02:08.678452 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.678425    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77919191-de58-4fb2-9f9e-a935562d5521-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "77919191-de58-4fb2-9f9e-a935562d5521" (UID: "77919191-de58-4fb2-9f9e-a935562d5521"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:02:08.679607 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.679571    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "77919191-de58-4fb2-9f9e-a935562d5521" (UID: "77919191-de58-4fb2-9f9e-a935562d5521"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:02:08.680136 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.680103    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77919191-de58-4fb2-9f9e-a935562d5521-kube-api-access-fw4mh" (OuterVolumeSpecName: "kube-api-access-fw4mh") pod "77919191-de58-4fb2-9f9e-a935562d5521" (UID: "77919191-de58-4fb2-9f9e-a935562d5521"). InnerVolumeSpecName "kube-api-access-fw4mh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:02:08.680234 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.680134    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "77919191-de58-4fb2-9f9e-a935562d5521" (UID: "77919191-de58-4fb2-9f9e-a935562d5521"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:02:08.680459 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.680418    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "77919191-de58-4fb2-9f9e-a935562d5521" (UID: "77919191-de58-4fb2-9f9e-a935562d5521"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:02:08.680459 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.680436    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-config-volume" (OuterVolumeSpecName: "config-volume") pod "77919191-de58-4fb2-9f9e-a935562d5521" (UID: "77919191-de58-4fb2-9f9e-a935562d5521"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:02:08.680631 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.680581    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "77919191-de58-4fb2-9f9e-a935562d5521" (UID: "77919191-de58-4fb2-9f9e-a935562d5521"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:02:08.685399 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.685327    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "77919191-de58-4fb2-9f9e-a935562d5521" (UID: "77919191-de58-4fb2-9f9e-a935562d5521"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:02:08.691929 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.691900    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-web-config" (OuterVolumeSpecName: "web-config") pod "77919191-de58-4fb2-9f9e-a935562d5521" (UID: "77919191-de58-4fb2-9f9e-a935562d5521"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:02:08.777339 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.777302    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/77919191-de58-4fb2-9f9e-a935562d5521-config-out\") pod \"77919191-de58-4fb2-9f9e-a935562d5521\" (UID: \"77919191-de58-4fb2-9f9e-a935562d5521\") "
Jul 10 23:02:08.777563 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.777504    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fw4mh\" (UniqueName: \"kubernetes.io/projected/77919191-de58-4fb2-9f9e-a935562d5521-kube-api-access-fw4mh\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:08.777563 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.777524    2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/77919191-de58-4fb2-9f9e-a935562d5521-tls-assets\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:08.777563 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.777539    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:08.777563 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.777551    2563 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-config-volume\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:08.777563 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.777563    2563 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-cluster-tls-config\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:08.777775 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.777575    2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-web-config\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:08.777775 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.777590    2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/77919191-de58-4fb2-9f9e-a935562d5521-alertmanager-main-db\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:08.777775 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.777603    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77919191-de58-4fb2-9f9e-a935562d5521-metrics-client-ca\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:08.777775 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.777617    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:08.777775 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.777632    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:08.777775 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.777647    2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/77919191-de58-4fb2-9f9e-a935562d5521-secret-alertmanager-main-tls\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:08.779538 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.779513    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77919191-de58-4fb2-9f9e-a935562d5521-config-out" (OuterVolumeSpecName: "config-out") pod "77919191-de58-4fb2-9f9e-a935562d5521" (UID: "77919191-de58-4fb2-9f9e-a935562d5521"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:02:08.878996 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:08.878965    2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/77919191-de58-4fb2-9f9e-a935562d5521-config-out\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:09.130526 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.130492    2563 generic.go:356] "Generic (PLEG): container finished" podID="77919191-de58-4fb2-9f9e-a935562d5521" containerID="6e8728fa21520b8a0e04a83d625b12cb312d3e16f28f24078ffc8d27a5516a52" exitCode=0
Jul 10 23:02:09.130526 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.130521    2563 generic.go:356] "Generic (PLEG): container finished" podID="77919191-de58-4fb2-9f9e-a935562d5521" containerID="22111a6372a5eae53c7e979b66e2ff1ec042fe8df049d2812d3de3c659454fad" exitCode=0
Jul 10 23:02:09.130746 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.130612    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77919191-de58-4fb2-9f9e-a935562d5521","Type":"ContainerDied","Data":"6e8728fa21520b8a0e04a83d625b12cb312d3e16f28f24078ffc8d27a5516a52"}
Jul 10 23:02:09.130746 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.130655    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.130746 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.130703    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77919191-de58-4fb2-9f9e-a935562d5521","Type":"ContainerDied","Data":"22111a6372a5eae53c7e979b66e2ff1ec042fe8df049d2812d3de3c659454fad"}
Jul 10 23:02:09.130746 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.130733    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"77919191-de58-4fb2-9f9e-a935562d5521","Type":"ContainerDied","Data":"c932cbe3c986b1f950bd1e1bd70c05c2a77602e9b0d5a86171186e43b2c6b959"}
Jul 10 23:02:09.131444 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.130775    2563 scope.go:117] "RemoveContainer" containerID="45fb8e20ea3a031919bca3263ed82a7a27e76b650876ce42bd5852df98b3fd22"
Jul 10 23:02:09.140120 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.140100    2563 scope.go:117] "RemoveContainer" containerID="6e8728fa21520b8a0e04a83d625b12cb312d3e16f28f24078ffc8d27a5516a52"
Jul 10 23:02:09.148108 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.148086    2563 scope.go:117] "RemoveContainer" containerID="922b41c1fd980087ab036120dcc34a630b72070ca4b3a8b48609b9f6cf19b915"
Jul 10 23:02:09.154119 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.154090    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 23:02:09.158130 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.158108    2563 scope.go:117] "RemoveContainer" containerID="22111a6372a5eae53c7e979b66e2ff1ec042fe8df049d2812d3de3c659454fad"
Jul 10 23:02:09.158255 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.158233    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 23:02:09.166059 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.166038    2563 scope.go:117] "RemoveContainer" containerID="8ee2b3bbe415ab1255bd492abbbad04acef8c7e18ebb8ffe90b45d942c465244"
Jul 10 23:02:09.174789 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.174769    2563 scope.go:117] "RemoveContainer" containerID="e2cac26339363173c1329c570509cfee3013d7d44445065ff6f06e4466a1e6c1"
Jul 10 23:02:09.182523 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.182494    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 23:02:09.182833 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.182819    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77919191-de58-4fb2-9f9e-a935562d5521" containerName="init-config-reloader"
Jul 10 23:02:09.182874 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.182836    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="77919191-de58-4fb2-9f9e-a935562d5521" containerName="init-config-reloader"
Jul 10 23:02:09.182874 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.182853    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77919191-de58-4fb2-9f9e-a935562d5521" containerName="kube-rbac-proxy"
Jul 10 23:02:09.182874 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.182859    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="77919191-de58-4fb2-9f9e-a935562d5521" containerName="kube-rbac-proxy"
Jul 10 23:02:09.182874 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.182866    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55ee7f77-d691-4e70-a9fc-35cb36f9c56d" containerName="console"
Jul 10 23:02:09.182874 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.182871    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="55ee7f77-d691-4e70-a9fc-35cb36f9c56d" containerName="console"
Jul 10 23:02:09.183021 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.182880    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77919191-de58-4fb2-9f9e-a935562d5521" containerName="alertmanager"
Jul 10 23:02:09.183021 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.182885    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="77919191-de58-4fb2-9f9e-a935562d5521" containerName="alertmanager"
Jul 10 23:02:09.183021 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.182891    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77919191-de58-4fb2-9f9e-a935562d5521" containerName="prom-label-proxy"
Jul 10 23:02:09.183021 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.182896    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="77919191-de58-4fb2-9f9e-a935562d5521" containerName="prom-label-proxy"
Jul 10 23:02:09.183021 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.182908    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77919191-de58-4fb2-9f9e-a935562d5521" containerName="kube-rbac-proxy-metric"
Jul 10 23:02:09.183021 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.182913    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="77919191-de58-4fb2-9f9e-a935562d5521" containerName="kube-rbac-proxy-metric"
Jul 10 23:02:09.183021 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.182925    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77919191-de58-4fb2-9f9e-a935562d5521" containerName="kube-rbac-proxy-web"
Jul 10 23:02:09.183021 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.182933    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="77919191-de58-4fb2-9f9e-a935562d5521" containerName="kube-rbac-proxy-web"
Jul 10 23:02:09.183021 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.182940    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="953d6354-ca9c-4dec-843e-ebf7497c4a95" containerName="console"
Jul 10 23:02:09.183021 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.182945    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="953d6354-ca9c-4dec-843e-ebf7497c4a95" containerName="console"
Jul 10 23:02:09.183021 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.182957    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77919191-de58-4fb2-9f9e-a935562d5521" containerName="config-reloader"
Jul 10 23:02:09.183021 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.182962    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="77919191-de58-4fb2-9f9e-a935562d5521" containerName="config-reloader"
Jul 10 23:02:09.183021 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.182968    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7f0af82-786b-4b48-be84-9ced1047ff52" containerName="registry"
Jul 10 23:02:09.183021 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.182973    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7f0af82-786b-4b48-be84-9ced1047ff52" containerName="registry"
Jul 10 23:02:09.183021 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.183024    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="77919191-de58-4fb2-9f9e-a935562d5521" containerName="prom-label-proxy"
Jul 10 23:02:09.183492 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.183034    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="77919191-de58-4fb2-9f9e-a935562d5521" containerName="config-reloader"
Jul 10 23:02:09.183492 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.183042    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="77919191-de58-4fb2-9f9e-a935562d5521" containerName="alertmanager"
Jul 10 23:02:09.183492 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.183047    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="77919191-de58-4fb2-9f9e-a935562d5521" containerName="kube-rbac-proxy-metric"
Jul 10 23:02:09.183492 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.183054    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="953d6354-ca9c-4dec-843e-ebf7497c4a95" containerName="console"
Jul 10 23:02:09.183492 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.183060    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="55ee7f77-d691-4e70-a9fc-35cb36f9c56d" containerName="console"
Jul 10 23:02:09.183492 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.183065    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="77919191-de58-4fb2-9f9e-a935562d5521" containerName="kube-rbac-proxy"
Jul 10 23:02:09.183492 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.183071    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7f0af82-786b-4b48-be84-9ced1047ff52" containerName="registry"
Jul 10 23:02:09.183492 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.183079    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="77919191-de58-4fb2-9f9e-a935562d5521" containerName="kube-rbac-proxy-web"
Jul 10 23:02:09.183492 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.183451    2563 scope.go:117] "RemoveContainer" containerID="6854cef15ecdb362e7a20271fe472db56baa82d505e1e345e36b4510a9eca6f6"
Jul 10 23:02:09.188329 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.188308    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.190924 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.190898    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 10 23:02:09.191101 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.190904    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 10 23:02:09.191176 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.191130    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 10 23:02:09.191176 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.190904    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 10 23:02:09.191176 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.191148    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-kgr2t\""
Jul 10 23:02:09.191317 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.190994    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 10 23:02:09.191317 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.190925    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 10 23:02:09.191317 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.191133    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 10 23:02:09.191492 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.191346    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 10 23:02:09.192878 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.192858    2563 scope.go:117] "RemoveContainer" containerID="45fb8e20ea3a031919bca3263ed82a7a27e76b650876ce42bd5852df98b3fd22"
Jul 10 23:02:09.193329 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:02:09.193245    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45fb8e20ea3a031919bca3263ed82a7a27e76b650876ce42bd5852df98b3fd22\": container with ID starting with 45fb8e20ea3a031919bca3263ed82a7a27e76b650876ce42bd5852df98b3fd22 not found: ID does not exist" containerID="45fb8e20ea3a031919bca3263ed82a7a27e76b650876ce42bd5852df98b3fd22"
Jul 10 23:02:09.193329 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.193303    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45fb8e20ea3a031919bca3263ed82a7a27e76b650876ce42bd5852df98b3fd22"} err="failed to get container status \"45fb8e20ea3a031919bca3263ed82a7a27e76b650876ce42bd5852df98b3fd22\": rpc error: code = NotFound desc = could not find container \"45fb8e20ea3a031919bca3263ed82a7a27e76b650876ce42bd5852df98b3fd22\": container with ID starting with 45fb8e20ea3a031919bca3263ed82a7a27e76b650876ce42bd5852df98b3fd22 not found: ID does not exist"
Jul 10 23:02:09.193329 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.193328    2563 scope.go:117] "RemoveContainer" containerID="6e8728fa21520b8a0e04a83d625b12cb312d3e16f28f24078ffc8d27a5516a52"
Jul 10 23:02:09.193726 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:02:09.193702    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e8728fa21520b8a0e04a83d625b12cb312d3e16f28f24078ffc8d27a5516a52\": container with ID starting with 6e8728fa21520b8a0e04a83d625b12cb312d3e16f28f24078ffc8d27a5516a52 not found: ID does not exist" containerID="6e8728fa21520b8a0e04a83d625b12cb312d3e16f28f24078ffc8d27a5516a52"
Jul 10 23:02:09.193855 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.193731    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e8728fa21520b8a0e04a83d625b12cb312d3e16f28f24078ffc8d27a5516a52"} err="failed to get container status \"6e8728fa21520b8a0e04a83d625b12cb312d3e16f28f24078ffc8d27a5516a52\": rpc error: code = NotFound desc = could not find container \"6e8728fa21520b8a0e04a83d625b12cb312d3e16f28f24078ffc8d27a5516a52\": container with ID starting with 6e8728fa21520b8a0e04a83d625b12cb312d3e16f28f24078ffc8d27a5516a52 not found: ID does not exist"
Jul 10 23:02:09.193855 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.193755    2563 scope.go:117] "RemoveContainer" containerID="922b41c1fd980087ab036120dcc34a630b72070ca4b3a8b48609b9f6cf19b915"
Jul 10 23:02:09.194095 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:02:09.194057    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"922b41c1fd980087ab036120dcc34a630b72070ca4b3a8b48609b9f6cf19b915\": container with ID starting with 922b41c1fd980087ab036120dcc34a630b72070ca4b3a8b48609b9f6cf19b915 not found: ID does not exist" containerID="922b41c1fd980087ab036120dcc34a630b72070ca4b3a8b48609b9f6cf19b915"
Jul 10 23:02:09.194095 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.194085    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"922b41c1fd980087ab036120dcc34a630b72070ca4b3a8b48609b9f6cf19b915"} err="failed to get container status \"922b41c1fd980087ab036120dcc34a630b72070ca4b3a8b48609b9f6cf19b915\": rpc error: code = NotFound desc = could not find container \"922b41c1fd980087ab036120dcc34a630b72070ca4b3a8b48609b9f6cf19b915\": container with ID starting with 922b41c1fd980087ab036120dcc34a630b72070ca4b3a8b48609b9f6cf19b915 not found: ID does not exist"
Jul 10 23:02:09.194230 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.194105    2563 scope.go:117] "RemoveContainer" containerID="22111a6372a5eae53c7e979b66e2ff1ec042fe8df049d2812d3de3c659454fad"
Jul 10 23:02:09.194399 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:02:09.194351    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22111a6372a5eae53c7e979b66e2ff1ec042fe8df049d2812d3de3c659454fad\": container with ID starting with 22111a6372a5eae53c7e979b66e2ff1ec042fe8df049d2812d3de3c659454fad not found: ID does not exist" containerID="22111a6372a5eae53c7e979b66e2ff1ec042fe8df049d2812d3de3c659454fad"
Jul 10 23:02:09.194523 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.194407    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22111a6372a5eae53c7e979b66e2ff1ec042fe8df049d2812d3de3c659454fad"} err="failed to get container status \"22111a6372a5eae53c7e979b66e2ff1ec042fe8df049d2812d3de3c659454fad\": rpc error: code = NotFound desc = could not find container \"22111a6372a5eae53c7e979b66e2ff1ec042fe8df049d2812d3de3c659454fad\": container with ID starting with 22111a6372a5eae53c7e979b66e2ff1ec042fe8df049d2812d3de3c659454fad not found: ID does not exist"
Jul 10 23:02:09.194523 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.194426    2563 scope.go:117] "RemoveContainer" containerID="8ee2b3bbe415ab1255bd492abbbad04acef8c7e18ebb8ffe90b45d942c465244"
Jul 10 23:02:09.194728 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:02:09.194703    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ee2b3bbe415ab1255bd492abbbad04acef8c7e18ebb8ffe90b45d942c465244\": container with ID starting with 8ee2b3bbe415ab1255bd492abbbad04acef8c7e18ebb8ffe90b45d942c465244 not found: ID does not exist" containerID="8ee2b3bbe415ab1255bd492abbbad04acef8c7e18ebb8ffe90b45d942c465244"
Jul 10 23:02:09.194832 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.194732    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ee2b3bbe415ab1255bd492abbbad04acef8c7e18ebb8ffe90b45d942c465244"} err="failed to get container status \"8ee2b3bbe415ab1255bd492abbbad04acef8c7e18ebb8ffe90b45d942c465244\": rpc error: code = NotFound desc = could not find container \"8ee2b3bbe415ab1255bd492abbbad04acef8c7e18ebb8ffe90b45d942c465244\": container with ID starting with 8ee2b3bbe415ab1255bd492abbbad04acef8c7e18ebb8ffe90b45d942c465244 not found: ID does not exist"
Jul 10 23:02:09.194832 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.194751    2563 scope.go:117] "RemoveContainer" containerID="e2cac26339363173c1329c570509cfee3013d7d44445065ff6f06e4466a1e6c1"
Jul 10 23:02:09.195027 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:02:09.195008    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2cac26339363173c1329c570509cfee3013d7d44445065ff6f06e4466a1e6c1\": container with ID starting with e2cac26339363173c1329c570509cfee3013d7d44445065ff6f06e4466a1e6c1 not found: ID does not exist" containerID="e2cac26339363173c1329c570509cfee3013d7d44445065ff6f06e4466a1e6c1"
Jul 10 23:02:09.195084 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.195031    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2cac26339363173c1329c570509cfee3013d7d44445065ff6f06e4466a1e6c1"} err="failed to get container status \"e2cac26339363173c1329c570509cfee3013d7d44445065ff6f06e4466a1e6c1\": rpc error: code = NotFound desc = could not find container \"e2cac26339363173c1329c570509cfee3013d7d44445065ff6f06e4466a1e6c1\": container with ID starting with e2cac26339363173c1329c570509cfee3013d7d44445065ff6f06e4466a1e6c1 not found: ID does not exist"
Jul 10 23:02:09.195084 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.195051    2563 scope.go:117] "RemoveContainer" containerID="6854cef15ecdb362e7a20271fe472db56baa82d505e1e345e36b4510a9eca6f6"
Jul 10 23:02:09.195415 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:02:09.195386    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6854cef15ecdb362e7a20271fe472db56baa82d505e1e345e36b4510a9eca6f6\": container with ID starting with 6854cef15ecdb362e7a20271fe472db56baa82d505e1e345e36b4510a9eca6f6 not found: ID does not exist" containerID="6854cef15ecdb362e7a20271fe472db56baa82d505e1e345e36b4510a9eca6f6"
Jul 10 23:02:09.195502 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.195422    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6854cef15ecdb362e7a20271fe472db56baa82d505e1e345e36b4510a9eca6f6"} err="failed to get container status \"6854cef15ecdb362e7a20271fe472db56baa82d505e1e345e36b4510a9eca6f6\": rpc error: code = NotFound desc = could not find container \"6854cef15ecdb362e7a20271fe472db56baa82d505e1e345e36b4510a9eca6f6\": container with ID starting with 6854cef15ecdb362e7a20271fe472db56baa82d505e1e345e36b4510a9eca6f6 not found: ID does not exist"
Jul 10 23:02:09.195502 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.195443    2563 scope.go:117] "RemoveContainer" containerID="45fb8e20ea3a031919bca3263ed82a7a27e76b650876ce42bd5852df98b3fd22"
Jul 10 23:02:09.196193 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.195798    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45fb8e20ea3a031919bca3263ed82a7a27e76b650876ce42bd5852df98b3fd22"} err="failed to get container status \"45fb8e20ea3a031919bca3263ed82a7a27e76b650876ce42bd5852df98b3fd22\": rpc error: code = NotFound desc = could not find container \"45fb8e20ea3a031919bca3263ed82a7a27e76b650876ce42bd5852df98b3fd22\": container with ID starting with 45fb8e20ea3a031919bca3263ed82a7a27e76b650876ce42bd5852df98b3fd22 not found: ID does not exist"
Jul 10 23:02:09.196193 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.195821    2563 scope.go:117] "RemoveContainer" containerID="6e8728fa21520b8a0e04a83d625b12cb312d3e16f28f24078ffc8d27a5516a52"
Jul 10 23:02:09.196193 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.196169    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e8728fa21520b8a0e04a83d625b12cb312d3e16f28f24078ffc8d27a5516a52"} err="failed to get container status \"6e8728fa21520b8a0e04a83d625b12cb312d3e16f28f24078ffc8d27a5516a52\": rpc error: code = NotFound desc = could not find container \"6e8728fa21520b8a0e04a83d625b12cb312d3e16f28f24078ffc8d27a5516a52\": container with ID starting with 6e8728fa21520b8a0e04a83d625b12cb312d3e16f28f24078ffc8d27a5516a52 not found: ID does not exist"
Jul 10 23:02:09.196193 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.196188    2563 scope.go:117] "RemoveContainer" containerID="922b41c1fd980087ab036120dcc34a630b72070ca4b3a8b48609b9f6cf19b915"
Jul 10 23:02:09.196460 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.196416    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 10 23:02:09.196659 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.196612    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"922b41c1fd980087ab036120dcc34a630b72070ca4b3a8b48609b9f6cf19b915"} err="failed to get container status \"922b41c1fd980087ab036120dcc34a630b72070ca4b3a8b48609b9f6cf19b915\": rpc error: code = NotFound desc = could not find container \"922b41c1fd980087ab036120dcc34a630b72070ca4b3a8b48609b9f6cf19b915\": container with ID starting with 922b41c1fd980087ab036120dcc34a630b72070ca4b3a8b48609b9f6cf19b915 not found: ID does not exist"
Jul 10 23:02:09.196659 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.196639    2563 scope.go:117] "RemoveContainer" containerID="22111a6372a5eae53c7e979b66e2ff1ec042fe8df049d2812d3de3c659454fad"
Jul 10 23:02:09.196928 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.196900    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22111a6372a5eae53c7e979b66e2ff1ec042fe8df049d2812d3de3c659454fad"} err="failed to get container status \"22111a6372a5eae53c7e979b66e2ff1ec042fe8df049d2812d3de3c659454fad\": rpc error: code = NotFound desc = could not find container \"22111a6372a5eae53c7e979b66e2ff1ec042fe8df049d2812d3de3c659454fad\": container with ID starting with 22111a6372a5eae53c7e979b66e2ff1ec042fe8df049d2812d3de3c659454fad not found: ID does not exist"
Jul 10 23:02:09.197011 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.196932    2563 scope.go:117] "RemoveContainer" containerID="8ee2b3bbe415ab1255bd492abbbad04acef8c7e18ebb8ffe90b45d942c465244"
Jul 10 23:02:09.197487 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.197431    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ee2b3bbe415ab1255bd492abbbad04acef8c7e18ebb8ffe90b45d942c465244"} err="failed to get container status \"8ee2b3bbe415ab1255bd492abbbad04acef8c7e18ebb8ffe90b45d942c465244\": rpc error: code = NotFound desc = could not find container \"8ee2b3bbe415ab1255bd492abbbad04acef8c7e18ebb8ffe90b45d942c465244\": container with ID starting with 8ee2b3bbe415ab1255bd492abbbad04acef8c7e18ebb8ffe90b45d942c465244 not found: ID does not exist"
Jul 10 23:02:09.197487 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.197455    2563 scope.go:117] "RemoveContainer" containerID="e2cac26339363173c1329c570509cfee3013d7d44445065ff6f06e4466a1e6c1"
Jul 10 23:02:09.197757 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.197738    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2cac26339363173c1329c570509cfee3013d7d44445065ff6f06e4466a1e6c1"} err="failed to get container status \"e2cac26339363173c1329c570509cfee3013d7d44445065ff6f06e4466a1e6c1\": rpc error: code = NotFound desc = could not find container \"e2cac26339363173c1329c570509cfee3013d7d44445065ff6f06e4466a1e6c1\": container with ID starting with e2cac26339363173c1329c570509cfee3013d7d44445065ff6f06e4466a1e6c1 not found: ID does not exist"
Jul 10 23:02:09.197757 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.197757    2563 scope.go:117] "RemoveContainer" containerID="6854cef15ecdb362e7a20271fe472db56baa82d505e1e345e36b4510a9eca6f6"
Jul 10 23:02:09.198057 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.198034    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6854cef15ecdb362e7a20271fe472db56baa82d505e1e345e36b4510a9eca6f6"} err="failed to get container status \"6854cef15ecdb362e7a20271fe472db56baa82d505e1e345e36b4510a9eca6f6\": rpc error: code = NotFound desc = could not find container \"6854cef15ecdb362e7a20271fe472db56baa82d505e1e345e36b4510a9eca6f6\": container with ID starting with 6854cef15ecdb362e7a20271fe472db56baa82d505e1e345e36b4510a9eca6f6 not found: ID does not exist"
Jul 10 23:02:09.203635 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.203603    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 23:02:09.280969 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.280928    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b9051d20-f72c-4818-9624-3c96503459a5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.281160 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.280974    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b9051d20-f72c-4818-9624-3c96503459a5-web-config\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.281160 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.281007    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b9051d20-f72c-4818-9624-3c96503459a5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.281160 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.281063    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b9051d20-f72c-4818-9624-3c96503459a5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.281160 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.281104    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b9051d20-f72c-4818-9624-3c96503459a5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.281160 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.281122    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9051d20-f72c-4818-9624-3c96503459a5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.281160 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.281150    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8nrq\" (UniqueName: \"kubernetes.io/projected/b9051d20-f72c-4818-9624-3c96503459a5-kube-api-access-w8nrq\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.281442 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.281182    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b9051d20-f72c-4818-9624-3c96503459a5-config-out\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.281442 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.281200    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b9051d20-f72c-4818-9624-3c96503459a5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.281442 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.281218    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b9051d20-f72c-4818-9624-3c96503459a5-config-volume\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.281442 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.281235    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b9051d20-f72c-4818-9624-3c96503459a5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.281442 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.281252    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b9051d20-f72c-4818-9624-3c96503459a5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.281442 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.281303    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9051d20-f72c-4818-9624-3c96503459a5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.382526 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.382422    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8nrq\" (UniqueName: \"kubernetes.io/projected/b9051d20-f72c-4818-9624-3c96503459a5-kube-api-access-w8nrq\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.382526 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.382473    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b9051d20-f72c-4818-9624-3c96503459a5-config-out\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.382526 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.382491    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b9051d20-f72c-4818-9624-3c96503459a5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.382747 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.382550    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b9051d20-f72c-4818-9624-3c96503459a5-config-volume\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.382747 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.382570    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b9051d20-f72c-4818-9624-3c96503459a5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.382840 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.382804    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b9051d20-f72c-4818-9624-3c96503459a5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.382901 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.382868    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9051d20-f72c-4818-9624-3c96503459a5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.382953 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.382925    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b9051d20-f72c-4818-9624-3c96503459a5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.383002 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.382957    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b9051d20-f72c-4818-9624-3c96503459a5-web-config\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.383002 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.382989    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b9051d20-f72c-4818-9624-3c96503459a5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.383096 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.383020    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b9051d20-f72c-4818-9624-3c96503459a5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.383096 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.383080    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b9051d20-f72c-4818-9624-3c96503459a5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.383195 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.383107    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9051d20-f72c-4818-9624-3c96503459a5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.384416 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.384066    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b9051d20-f72c-4818-9624-3c96503459a5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.384416 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.384233    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9051d20-f72c-4818-9624-3c96503459a5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.385780 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.385745    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b9051d20-f72c-4818-9624-3c96503459a5-config-out\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.385890 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.385831    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b9051d20-f72c-4818-9624-3c96503459a5-config-volume\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.385890 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.385837    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b9051d20-f72c-4818-9624-3c96503459a5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.386033 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.385924    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b9051d20-f72c-4818-9624-3c96503459a5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.386033 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.385972    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b9051d20-f72c-4818-9624-3c96503459a5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.386142 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.386111    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b9051d20-f72c-4818-9624-3c96503459a5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.386318 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.386300    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b9051d20-f72c-4818-9624-3c96503459a5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.386698 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.386680    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b9051d20-f72c-4818-9624-3c96503459a5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.386833 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.386817    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b9051d20-f72c-4818-9624-3c96503459a5-web-config\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.387795 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.387778    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b9051d20-f72c-4818-9624-3c96503459a5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.390167 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.390140    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8nrq\" (UniqueName: \"kubernetes.io/projected/b9051d20-f72c-4818-9624-3c96503459a5-kube-api-access-w8nrq\") pod \"alertmanager-main-0\" (UID: \"b9051d20-f72c-4818-9624-3c96503459a5\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.500573 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.500515    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 23:02:09.557111 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.557077    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77919191-de58-4fb2-9f9e-a935562d5521" path="/var/lib/kubelet/pods/77919191-de58-4fb2-9f9e-a935562d5521/volumes"
Jul 10 23:02:09.653296 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:09.653152    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 23:02:09.655784 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:02:09.655749    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9051d20_f72c_4818_9624_3c96503459a5.slice/crio-e0cb0626bcbbdae2fbad416ab9731f6c7b754f4c3751b3aa4d108bae33911ff4 WatchSource:0}: Error finding container e0cb0626bcbbdae2fbad416ab9731f6c7b754f4c3751b3aa4d108bae33911ff4: Status 404 returned error can't find the container with id e0cb0626bcbbdae2fbad416ab9731f6c7b754f4c3751b3aa4d108bae33911ff4
Jul 10 23:02:10.137135 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:10.137099    2563 generic.go:356] "Generic (PLEG): container finished" podID="b9051d20-f72c-4818-9624-3c96503459a5" containerID="aecffa438c7c05597f55cd4a8779eec500b32e1c85c42a12969187a7e3b2e9f7" exitCode=0
Jul 10 23:02:10.137348 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:10.137161    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b9051d20-f72c-4818-9624-3c96503459a5","Type":"ContainerDied","Data":"aecffa438c7c05597f55cd4a8779eec500b32e1c85c42a12969187a7e3b2e9f7"}
Jul 10 23:02:10.137348 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:10.137233    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b9051d20-f72c-4818-9624-3c96503459a5","Type":"ContainerStarted","Data":"e0cb0626bcbbdae2fbad416ab9731f6c7b754f4c3751b3aa4d108bae33911ff4"}
Jul 10 23:02:10.865654 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:10.865513    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:02:10.966543 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:10.966503    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:02:11.145338 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:11.145247    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b9051d20-f72c-4818-9624-3c96503459a5","Type":"ContainerStarted","Data":"2d2c70c7d4d98420193b6b12d94f83e33c5ca00bb14e0ec2d069ebe455cbaa48"}
Jul 10 23:02:11.145338 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:11.145330    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b9051d20-f72c-4818-9624-3c96503459a5","Type":"ContainerStarted","Data":"572c93d3009340af0d112379908849afc9e7979d6685f2726aa2d4586a99e97b"}
Jul 10 23:02:11.145553 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:11.145396    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b9051d20-f72c-4818-9624-3c96503459a5","Type":"ContainerStarted","Data":"adb920960e68b0d3b336e3d36512c2c87401514931d7ce6d4df7269f9d84e8b7"}
Jul 10 23:02:11.145553 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:11.145440    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b9051d20-f72c-4818-9624-3c96503459a5","Type":"ContainerStarted","Data":"0172e02fd7127bb9f6358fecdb1880ff9e0a18325e2ab31fb87e11cf352f1cbe"}
Jul 10 23:02:11.145553 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:11.145489    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b9051d20-f72c-4818-9624-3c96503459a5","Type":"ContainerStarted","Data":"0baeb2a147ed4a3baf0bc346ca86dfe89400a0fba28e71ccd354ff627ef629d9"}
Jul 10 23:02:11.145553 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:11.145529    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b9051d20-f72c-4818-9624-3c96503459a5","Type":"ContainerStarted","Data":"872049386a881e1121ee09375e71ecdda67798cfa03f6661f14345f1dae37a6a"}
Jul 10 23:02:11.173617 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:11.173566    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.158436643 podStartE2EDuration="2.173550373s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.11373ms" isStatefulPod=false podCreationTimestamp="2026-07-10 23:02:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:02:11.172655484 +0000 UTC m=+392.059391627" watchObservedRunningTime="2026-07-10 23:02:11.173550373 +0000 UTC m=+392.060286510"
Jul 10 23:02:11.565933 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:11.565893    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 23:02:11.566244 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:11.566192    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2bddddf0-512e-466c-9327-b88418a330e4" containerName="prometheus" containerID="cri-o://bd22df776c0480e0c627cd65c3b42d3b92d7e56284f4baf81d73a877c9375cf2" gracePeriod=600
Jul 10 23:02:11.566336 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:11.566254    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2bddddf0-512e-466c-9327-b88418a330e4" containerName="config-reloader" containerID="cri-o://e583b6b6ab7395735ec3bab03fbfc072be6db48641354ee2e9050dfe5d3476ea" gracePeriod=600
Jul 10 23:02:11.566336 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:11.566221    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2bddddf0-512e-466c-9327-b88418a330e4" containerName="kube-rbac-proxy" containerID="cri-o://c61d4eaf115c0368a8e0b831824ea8cc7dfc7496c939d1e1a9e086f7b99356b3" gracePeriod=600
Jul 10 23:02:11.566492 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:11.566243    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2bddddf0-512e-466c-9327-b88418a330e4" containerName="kube-rbac-proxy-thanos" containerID="cri-o://34bc5ff54638f3356f597ccd349797c3127536e38919b5418d8a8b8f9dd37e1f" gracePeriod=600
Jul 10 23:02:11.566492 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:11.566254    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2bddddf0-512e-466c-9327-b88418a330e4" containerName="thanos-sidecar" containerID="cri-o://7a344f68c31b558b8d22905866867ef06d6793dca58c92d6bb8b16df01b410a2" gracePeriod=600
Jul 10 23:02:11.566601 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:11.566555    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="2bddddf0-512e-466c-9327-b88418a330e4" containerName="kube-rbac-proxy-web" containerID="cri-o://2108db9674798ab3098c8f7d67411053da50c56afc713d4d7b04926403a5dc9e" gracePeriod=600
Jul 10 23:02:12.153856 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.153821    2563 generic.go:356] "Generic (PLEG): container finished" podID="2bddddf0-512e-466c-9327-b88418a330e4" containerID="34bc5ff54638f3356f597ccd349797c3127536e38919b5418d8a8b8f9dd37e1f" exitCode=0
Jul 10 23:02:12.153856 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.153847    2563 generic.go:356] "Generic (PLEG): container finished" podID="2bddddf0-512e-466c-9327-b88418a330e4" containerID="c61d4eaf115c0368a8e0b831824ea8cc7dfc7496c939d1e1a9e086f7b99356b3" exitCode=0
Jul 10 23:02:12.153856 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.153854    2563 generic.go:356] "Generic (PLEG): container finished" podID="2bddddf0-512e-466c-9327-b88418a330e4" containerID="7a344f68c31b558b8d22905866867ef06d6793dca58c92d6bb8b16df01b410a2" exitCode=0
Jul 10 23:02:12.153856 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.153860    2563 generic.go:356] "Generic (PLEG): container finished" podID="2bddddf0-512e-466c-9327-b88418a330e4" containerID="e583b6b6ab7395735ec3bab03fbfc072be6db48641354ee2e9050dfe5d3476ea" exitCode=0
Jul 10 23:02:12.153856 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.153865    2563 generic.go:356] "Generic (PLEG): container finished" podID="2bddddf0-512e-466c-9327-b88418a330e4" containerID="bd22df776c0480e0c627cd65c3b42d3b92d7e56284f4baf81d73a877c9375cf2" exitCode=0
Jul 10 23:02:12.154356 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.153889    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bddddf0-512e-466c-9327-b88418a330e4","Type":"ContainerDied","Data":"34bc5ff54638f3356f597ccd349797c3127536e38919b5418d8a8b8f9dd37e1f"}
Jul 10 23:02:12.154356 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.153959    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bddddf0-512e-466c-9327-b88418a330e4","Type":"ContainerDied","Data":"c61d4eaf115c0368a8e0b831824ea8cc7dfc7496c939d1e1a9e086f7b99356b3"}
Jul 10 23:02:12.154356 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.153987    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bddddf0-512e-466c-9327-b88418a330e4","Type":"ContainerDied","Data":"7a344f68c31b558b8d22905866867ef06d6793dca58c92d6bb8b16df01b410a2"}
Jul 10 23:02:12.154356 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.154012    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bddddf0-512e-466c-9327-b88418a330e4","Type":"ContainerDied","Data":"e583b6b6ab7395735ec3bab03fbfc072be6db48641354ee2e9050dfe5d3476ea"}
Jul 10 23:02:12.154356 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.154035    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bddddf0-512e-466c-9327-b88418a330e4","Type":"ContainerDied","Data":"bd22df776c0480e0c627cd65c3b42d3b92d7e56284f4baf81d73a877c9375cf2"}
Jul 10 23:02:12.826328 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.826297    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:12.914645 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.914547    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-trusted-ca-bundle\") pod \"2bddddf0-512e-466c-9327-b88418a330e4\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") "
Jul 10 23:02:12.914645 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.914589    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2bddddf0-512e-466c-9327-b88418a330e4-config-out\") pod \"2bddddf0-512e-466c-9327-b88418a330e4\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") "
Jul 10 23:02:12.914645 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.914612    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-web-config\") pod \"2bddddf0-512e-466c-9327-b88418a330e4\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") "
Jul 10 23:02:12.914645 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.914629    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-k8s-db\") pod \"2bddddf0-512e-466c-9327-b88418a330e4\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") "
Jul 10 23:02:12.914970 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.914651    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"2bddddf0-512e-466c-9327-b88418a330e4\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") "
Jul 10 23:02:12.914970 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.914750    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"2bddddf0-512e-466c-9327-b88418a330e4\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") "
Jul 10 23:02:12.914970 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.914805    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-prometheus-k8s-tls\") pod \"2bddddf0-512e-466c-9327-b88418a330e4\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") "
Jul 10 23:02:12.914970 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.914833    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-configmap-serving-certs-ca-bundle\") pod \"2bddddf0-512e-466c-9327-b88418a330e4\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") "
Jul 10 23:02:12.914970 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.914867    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2bddddf0-512e-466c-9327-b88418a330e4-tls-assets\") pod \"2bddddf0-512e-466c-9327-b88418a330e4\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") "
Jul 10 23:02:12.914970 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.914911    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-k8s-rulefiles-1\") pod \"2bddddf0-512e-466c-9327-b88418a330e4\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") "
Jul 10 23:02:12.914970 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.914944    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-grpc-tls\") pod \"2bddddf0-512e-466c-9327-b88418a330e4\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") "
Jul 10 23:02:12.914970 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.914968    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qhrh\" (UniqueName: \"kubernetes.io/projected/2bddddf0-512e-466c-9327-b88418a330e4-kube-api-access-5qhrh\") pod \"2bddddf0-512e-466c-9327-b88418a330e4\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") "
Jul 10 23:02:12.915392 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.915032    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-kube-rbac-proxy\") pod \"2bddddf0-512e-466c-9327-b88418a330e4\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") "
Jul 10 23:02:12.915392 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.915056    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "2bddddf0-512e-466c-9327-b88418a330e4" (UID: "2bddddf0-512e-466c-9327-b88418a330e4"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:02:12.915392 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.915069    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-metrics-client-certs\") pod \"2bddddf0-512e-466c-9327-b88418a330e4\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") "
Jul 10 23:02:12.915392 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.915148    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-k8s-rulefiles-0\") pod \"2bddddf0-512e-466c-9327-b88418a330e4\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") "
Jul 10 23:02:12.915392 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.915271    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-thanos-prometheus-http-client-file\") pod \"2bddddf0-512e-466c-9327-b88418a330e4\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") "
Jul 10 23:02:12.915392 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.915311    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-configmap-metrics-client-ca\") pod \"2bddddf0-512e-466c-9327-b88418a330e4\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") "
Jul 10 23:02:12.915392 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.915335    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-config\") pod \"2bddddf0-512e-466c-9327-b88418a330e4\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") "
Jul 10 23:02:12.915742 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.915433    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-configmap-kubelet-serving-ca-bundle\") pod \"2bddddf0-512e-466c-9327-b88418a330e4\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") "
Jul 10 23:02:12.915742 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.915574    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-k8s-rulefiles-2\") pod \"2bddddf0-512e-466c-9327-b88418a330e4\" (UID: \"2bddddf0-512e-466c-9327-b88418a330e4\") "
Jul 10 23:02:12.915992 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.915964    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "2bddddf0-512e-466c-9327-b88418a330e4" (UID: "2bddddf0-512e-466c-9327-b88418a330e4"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:02:12.916070 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.915973    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "2bddddf0-512e-466c-9327-b88418a330e4" (UID: "2bddddf0-512e-466c-9327-b88418a330e4"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:02:12.916621 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.916441    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "2bddddf0-512e-466c-9327-b88418a330e4" (UID: "2bddddf0-512e-466c-9327-b88418a330e4"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:02:12.916621 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.916504    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "2bddddf0-512e-466c-9327-b88418a330e4" (UID: "2bddddf0-512e-466c-9327-b88418a330e4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:02:12.916995 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.916924    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "2bddddf0-512e-466c-9327-b88418a330e4" (UID: "2bddddf0-512e-466c-9327-b88418a330e4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:02:12.918094 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.918062    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "2bddddf0-512e-466c-9327-b88418a330e4" (UID: "2bddddf0-512e-466c-9327-b88418a330e4"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:02:12.918518 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.918152    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "2bddddf0-512e-466c-9327-b88418a330e4" (UID: "2bddddf0-512e-466c-9327-b88418a330e4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:02:12.918518 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.918487    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "2bddddf0-512e-466c-9327-b88418a330e4" (UID: "2bddddf0-512e-466c-9327-b88418a330e4"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:02:12.918686 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.918612    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "2bddddf0-512e-466c-9327-b88418a330e4" (UID: "2bddddf0-512e-466c-9327-b88418a330e4"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:02:12.918748 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.918691    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "2bddddf0-512e-466c-9327-b88418a330e4" (UID: "2bddddf0-512e-466c-9327-b88418a330e4"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:02:12.919090 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.919070    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:12.919238 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.919223    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-configmap-metrics-client-ca\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:12.919336 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.919322    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:12.919466 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.919449    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:12.919583 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.919569    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-trusted-ca-bundle\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:12.919708 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.919688    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-k8s-db\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:12.919857 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.919711    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:12.919857 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.919727    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:12.919857 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.919350    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "2bddddf0-512e-466c-9327-b88418a330e4" (UID: "2bddddf0-512e-466c-9327-b88418a330e4"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:02:12.919857 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.919640    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "2bddddf0-512e-466c-9327-b88418a330e4" (UID: "2bddddf0-512e-466c-9327-b88418a330e4"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:02:12.919857 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.919741    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-prometheus-k8s-tls\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:12.919857 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.919786    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:12.919857 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.919815    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2bddddf0-512e-466c-9327-b88418a330e4-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:12.920178 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.919965    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "2bddddf0-512e-466c-9327-b88418a330e4" (UID: "2bddddf0-512e-466c-9327-b88418a330e4"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:02:12.920178 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.920162    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bddddf0-512e-466c-9327-b88418a330e4-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "2bddddf0-512e-466c-9327-b88418a330e4" (UID: "2bddddf0-512e-466c-9327-b88418a330e4"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:02:12.920431 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.920408    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "2bddddf0-512e-466c-9327-b88418a330e4" (UID: "2bddddf0-512e-466c-9327-b88418a330e4"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:02:12.920827 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.920804    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bddddf0-512e-466c-9327-b88418a330e4-kube-api-access-5qhrh" (OuterVolumeSpecName: "kube-api-access-5qhrh") pod "2bddddf0-512e-466c-9327-b88418a330e4" (UID: "2bddddf0-512e-466c-9327-b88418a330e4"). InnerVolumeSpecName "kube-api-access-5qhrh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:02:12.920972 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.920951    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-config" (OuterVolumeSpecName: "config") pod "2bddddf0-512e-466c-9327-b88418a330e4" (UID: "2bddddf0-512e-466c-9327-b88418a330e4"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:02:12.920972 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.920957    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bddddf0-512e-466c-9327-b88418a330e4-config-out" (OuterVolumeSpecName: "config-out") pod "2bddddf0-512e-466c-9327-b88418a330e4" (UID: "2bddddf0-512e-466c-9327-b88418a330e4"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:02:12.932154 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:12.932122    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-web-config" (OuterVolumeSpecName: "web-config") pod "2bddddf0-512e-466c-9327-b88418a330e4" (UID: "2bddddf0-512e-466c-9327-b88418a330e4"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:02:13.020767 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.020712    2563 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-grpc-tls\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:13.020767 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.020757    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qhrh\" (UniqueName: \"kubernetes.io/projected/2bddddf0-512e-466c-9327-b88418a330e4-kube-api-access-5qhrh\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:13.020767 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.020768    2563 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-kube-rbac-proxy\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:13.020767 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.020777    2563 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-secret-metrics-client-certs\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:13.020767 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.020790    2563 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-thanos-prometheus-http-client-file\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:13.021060 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.020799    2563 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-config\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:13.021060 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.020807    2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2bddddf0-512e-466c-9327-b88418a330e4-config-out\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:13.021060 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.020817    2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2bddddf0-512e-466c-9327-b88418a330e4-web-config\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:13.021060 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.020825    2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2bddddf0-512e-466c-9327-b88418a330e4-tls-assets\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:02:13.161833 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.161803    2563 generic.go:356] "Generic (PLEG): container finished" podID="2bddddf0-512e-466c-9327-b88418a330e4" containerID="2108db9674798ab3098c8f7d67411053da50c56afc713d4d7b04926403a5dc9e" exitCode=0
Jul 10 23:02:13.162227 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.161852    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bddddf0-512e-466c-9327-b88418a330e4","Type":"ContainerDied","Data":"2108db9674798ab3098c8f7d67411053da50c56afc713d4d7b04926403a5dc9e"}
Jul 10 23:02:13.162227 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.161884    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.162227 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.161910    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2bddddf0-512e-466c-9327-b88418a330e4","Type":"ContainerDied","Data":"1df118154b39623f1510b42496c4fe923909a84f37948c073752758630cea40d"}
Jul 10 23:02:13.162227 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.161945    2563 scope.go:117] "RemoveContainer" containerID="34bc5ff54638f3356f597ccd349797c3127536e38919b5418d8a8b8f9dd37e1f"
Jul 10 23:02:13.170339 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.170274    2563 scope.go:117] "RemoveContainer" containerID="c61d4eaf115c0368a8e0b831824ea8cc7dfc7496c939d1e1a9e086f7b99356b3"
Jul 10 23:02:13.178253 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.178226    2563 scope.go:117] "RemoveContainer" containerID="2108db9674798ab3098c8f7d67411053da50c56afc713d4d7b04926403a5dc9e"
Jul 10 23:02:13.185386 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.185338    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 23:02:13.186603 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.186591    2563 scope.go:117] "RemoveContainer" containerID="7a344f68c31b558b8d22905866867ef06d6793dca58c92d6bb8b16df01b410a2"
Jul 10 23:02:13.190670 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.190643    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 23:02:13.194875 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.194854    2563 scope.go:117] "RemoveContainer" containerID="e583b6b6ab7395735ec3bab03fbfc072be6db48641354ee2e9050dfe5d3476ea"
Jul 10 23:02:13.202799 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.202778    2563 scope.go:117] "RemoveContainer" containerID="bd22df776c0480e0c627cd65c3b42d3b92d7e56284f4baf81d73a877c9375cf2"
Jul 10 23:02:13.211378 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.211336    2563 scope.go:117] "RemoveContainer" containerID="e5b73f0955a4c5c82ab1d7fd921e504a83792b82dab39f3a5e10d23420d1ebcd"
Jul 10 23:02:13.214263 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.214238    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 23:02:13.214587 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.214572    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bddddf0-512e-466c-9327-b88418a330e4" containerName="kube-rbac-proxy-web"
Jul 10 23:02:13.214646 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.214591    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bddddf0-512e-466c-9327-b88418a330e4" containerName="kube-rbac-proxy-web"
Jul 10 23:02:13.214646 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.214601    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bddddf0-512e-466c-9327-b88418a330e4" containerName="config-reloader"
Jul 10 23:02:13.214646 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.214606    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bddddf0-512e-466c-9327-b88418a330e4" containerName="config-reloader"
Jul 10 23:02:13.214646 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.214620    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bddddf0-512e-466c-9327-b88418a330e4" containerName="prometheus"
Jul 10 23:02:13.214646 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.214625    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bddddf0-512e-466c-9327-b88418a330e4" containerName="prometheus"
Jul 10 23:02:13.214646 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.214634    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bddddf0-512e-466c-9327-b88418a330e4" containerName="kube-rbac-proxy"
Jul 10 23:02:13.214646 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.214639    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bddddf0-512e-466c-9327-b88418a330e4" containerName="kube-rbac-proxy"
Jul 10 23:02:13.214646 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.214645    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bddddf0-512e-466c-9327-b88418a330e4" containerName="kube-rbac-proxy-thanos"
Jul 10 23:02:13.214865 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.214650    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bddddf0-512e-466c-9327-b88418a330e4" containerName="kube-rbac-proxy-thanos"
Jul 10 23:02:13.214865 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.214658    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bddddf0-512e-466c-9327-b88418a330e4" containerName="thanos-sidecar"
Jul 10 23:02:13.214865 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.214664    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bddddf0-512e-466c-9327-b88418a330e4" containerName="thanos-sidecar"
Jul 10 23:02:13.214865 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.214677    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bddddf0-512e-466c-9327-b88418a330e4" containerName="init-config-reloader"
Jul 10 23:02:13.214865 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.214682    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bddddf0-512e-466c-9327-b88418a330e4" containerName="init-config-reloader"
Jul 10 23:02:13.214865 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.214730    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bddddf0-512e-466c-9327-b88418a330e4" containerName="config-reloader"
Jul 10 23:02:13.214865 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.214737    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bddddf0-512e-466c-9327-b88418a330e4" containerName="kube-rbac-proxy-web"
Jul 10 23:02:13.214865 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.214745    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bddddf0-512e-466c-9327-b88418a330e4" containerName="kube-rbac-proxy"
Jul 10 23:02:13.214865 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.214750    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bddddf0-512e-466c-9327-b88418a330e4" containerName="prometheus"
Jul 10 23:02:13.214865 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.214756    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bddddf0-512e-466c-9327-b88418a330e4" containerName="thanos-sidecar"
Jul 10 23:02:13.214865 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.214762    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bddddf0-512e-466c-9327-b88418a330e4" containerName="kube-rbac-proxy-thanos"
Jul 10 23:02:13.220316 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.220289    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.220437 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.220418    2563 scope.go:117] "RemoveContainer" containerID="34bc5ff54638f3356f597ccd349797c3127536e38919b5418d8a8b8f9dd37e1f"
Jul 10 23:02:13.221474 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:02:13.220857    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34bc5ff54638f3356f597ccd349797c3127536e38919b5418d8a8b8f9dd37e1f\": container with ID starting with 34bc5ff54638f3356f597ccd349797c3127536e38919b5418d8a8b8f9dd37e1f not found: ID does not exist" containerID="34bc5ff54638f3356f597ccd349797c3127536e38919b5418d8a8b8f9dd37e1f"
Jul 10 23:02:13.221474 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.221262    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34bc5ff54638f3356f597ccd349797c3127536e38919b5418d8a8b8f9dd37e1f"} err="failed to get container status \"34bc5ff54638f3356f597ccd349797c3127536e38919b5418d8a8b8f9dd37e1f\": rpc error: code = NotFound desc = could not find container \"34bc5ff54638f3356f597ccd349797c3127536e38919b5418d8a8b8f9dd37e1f\": container with ID starting with 34bc5ff54638f3356f597ccd349797c3127536e38919b5418d8a8b8f9dd37e1f not found: ID does not exist"
Jul 10 23:02:13.221474 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.221293    2563 scope.go:117] "RemoveContainer" containerID="c61d4eaf115c0368a8e0b831824ea8cc7dfc7496c939d1e1a9e086f7b99356b3"
Jul 10 23:02:13.227030 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:02:13.226988    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c61d4eaf115c0368a8e0b831824ea8cc7dfc7496c939d1e1a9e086f7b99356b3\": container with ID starting with c61d4eaf115c0368a8e0b831824ea8cc7dfc7496c939d1e1a9e086f7b99356b3 not found: ID does not exist" containerID="c61d4eaf115c0368a8e0b831824ea8cc7dfc7496c939d1e1a9e086f7b99356b3"
Jul 10 23:02:13.227260 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.227039    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c61d4eaf115c0368a8e0b831824ea8cc7dfc7496c939d1e1a9e086f7b99356b3"} err="failed to get container status \"c61d4eaf115c0368a8e0b831824ea8cc7dfc7496c939d1e1a9e086f7b99356b3\": rpc error: code = NotFound desc = could not find container \"c61d4eaf115c0368a8e0b831824ea8cc7dfc7496c939d1e1a9e086f7b99356b3\": container with ID starting with c61d4eaf115c0368a8e0b831824ea8cc7dfc7496c939d1e1a9e086f7b99356b3 not found: ID does not exist"
Jul 10 23:02:13.227260 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.227065    2563 scope.go:117] "RemoveContainer" containerID="2108db9674798ab3098c8f7d67411053da50c56afc713d4d7b04926403a5dc9e"
Jul 10 23:02:13.227432 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.227416    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 10 23:02:13.227499 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.227429    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 10 23:02:13.227499 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.227416    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 10 23:02:13.227608 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.227507    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 10 23:02:13.227608 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.227521    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 10 23:02:13.227608 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.227416    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 10 23:02:13.228012 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.227766    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 10 23:02:13.228012 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.227791    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 10 23:02:13.228012 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.227810    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 10 23:02:13.228012 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.227814    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 10 23:02:13.228012 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:02:13.227892    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2108db9674798ab3098c8f7d67411053da50c56afc713d4d7b04926403a5dc9e\": container with ID starting with 2108db9674798ab3098c8f7d67411053da50c56afc713d4d7b04926403a5dc9e not found: ID does not exist" containerID="2108db9674798ab3098c8f7d67411053da50c56afc713d4d7b04926403a5dc9e"
Jul 10 23:02:13.228012 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.227912    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 10 23:02:13.228012 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.227921    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2108db9674798ab3098c8f7d67411053da50c56afc713d4d7b04926403a5dc9e"} err="failed to get container status \"2108db9674798ab3098c8f7d67411053da50c56afc713d4d7b04926403a5dc9e\": rpc error: code = NotFound desc = could not find container \"2108db9674798ab3098c8f7d67411053da50c56afc713d4d7b04926403a5dc9e\": container with ID starting with 2108db9674798ab3098c8f7d67411053da50c56afc713d4d7b04926403a5dc9e not found: ID does not exist"
Jul 10 23:02:13.228012 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.227914    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ch12p89tseoff\""
Jul 10 23:02:13.228012 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.227945    2563 scope.go:117] "RemoveContainer" containerID="7a344f68c31b558b8d22905866867ef06d6793dca58c92d6bb8b16df01b410a2"
Jul 10 23:02:13.228580 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:02:13.228290    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a344f68c31b558b8d22905866867ef06d6793dca58c92d6bb8b16df01b410a2\": container with ID starting with 7a344f68c31b558b8d22905866867ef06d6793dca58c92d6bb8b16df01b410a2 not found: ID does not exist" containerID="7a344f68c31b558b8d22905866867ef06d6793dca58c92d6bb8b16df01b410a2"
Jul 10 23:02:13.228580 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.228310    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-snfnn\""
Jul 10 23:02:13.228580 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.228315    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a344f68c31b558b8d22905866867ef06d6793dca58c92d6bb8b16df01b410a2"} err="failed to get container status \"7a344f68c31b558b8d22905866867ef06d6793dca58c92d6bb8b16df01b410a2\": rpc error: code = NotFound desc = could not find container \"7a344f68c31b558b8d22905866867ef06d6793dca58c92d6bb8b16df01b410a2\": container with ID starting with 7a344f68c31b558b8d22905866867ef06d6793dca58c92d6bb8b16df01b410a2 not found: ID does not exist"
Jul 10 23:02:13.228580 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.228334    2563 scope.go:117] "RemoveContainer" containerID="e583b6b6ab7395735ec3bab03fbfc072be6db48641354ee2e9050dfe5d3476ea"
Jul 10 23:02:13.228580 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.228322    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 10 23:02:13.228842 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:02:13.228666    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e583b6b6ab7395735ec3bab03fbfc072be6db48641354ee2e9050dfe5d3476ea\": container with ID starting with e583b6b6ab7395735ec3bab03fbfc072be6db48641354ee2e9050dfe5d3476ea not found: ID does not exist" containerID="e583b6b6ab7395735ec3bab03fbfc072be6db48641354ee2e9050dfe5d3476ea"
Jul 10 23:02:13.228842 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.228694    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e583b6b6ab7395735ec3bab03fbfc072be6db48641354ee2e9050dfe5d3476ea"} err="failed to get container status \"e583b6b6ab7395735ec3bab03fbfc072be6db48641354ee2e9050dfe5d3476ea\": rpc error: code = NotFound desc = could not find container \"e583b6b6ab7395735ec3bab03fbfc072be6db48641354ee2e9050dfe5d3476ea\": container with ID starting with e583b6b6ab7395735ec3bab03fbfc072be6db48641354ee2e9050dfe5d3476ea not found: ID does not exist"
Jul 10 23:02:13.228842 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.228713    2563 scope.go:117] "RemoveContainer" containerID="bd22df776c0480e0c627cd65c3b42d3b92d7e56284f4baf81d73a877c9375cf2"
Jul 10 23:02:13.228842 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.228764    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 10 23:02:13.229035 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:02:13.228981    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd22df776c0480e0c627cd65c3b42d3b92d7e56284f4baf81d73a877c9375cf2\": container with ID starting with bd22df776c0480e0c627cd65c3b42d3b92d7e56284f4baf81d73a877c9375cf2 not found: ID does not exist" containerID="bd22df776c0480e0c627cd65c3b42d3b92d7e56284f4baf81d73a877c9375cf2"
Jul 10 23:02:13.229035 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.229009    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd22df776c0480e0c627cd65c3b42d3b92d7e56284f4baf81d73a877c9375cf2"} err="failed to get container status \"bd22df776c0480e0c627cd65c3b42d3b92d7e56284f4baf81d73a877c9375cf2\": rpc error: code = NotFound desc = could not find container \"bd22df776c0480e0c627cd65c3b42d3b92d7e56284f4baf81d73a877c9375cf2\": container with ID starting with bd22df776c0480e0c627cd65c3b42d3b92d7e56284f4baf81d73a877c9375cf2 not found: ID does not exist"
Jul 10 23:02:13.229035 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.229029    2563 scope.go:117] "RemoveContainer" containerID="e5b73f0955a4c5c82ab1d7fd921e504a83792b82dab39f3a5e10d23420d1ebcd"
Jul 10 23:02:13.229421 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:02:13.229391    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5b73f0955a4c5c82ab1d7fd921e504a83792b82dab39f3a5e10d23420d1ebcd\": container with ID starting with e5b73f0955a4c5c82ab1d7fd921e504a83792b82dab39f3a5e10d23420d1ebcd not found: ID does not exist" containerID="e5b73f0955a4c5c82ab1d7fd921e504a83792b82dab39f3a5e10d23420d1ebcd"
Jul 10 23:02:13.229521 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.229428    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5b73f0955a4c5c82ab1d7fd921e504a83792b82dab39f3a5e10d23420d1ebcd"} err="failed to get container status \"e5b73f0955a4c5c82ab1d7fd921e504a83792b82dab39f3a5e10d23420d1ebcd\": rpc error: code = NotFound desc = could not find container \"e5b73f0955a4c5c82ab1d7fd921e504a83792b82dab39f3a5e10d23420d1ebcd\": container with ID starting with e5b73f0955a4c5c82ab1d7fd921e504a83792b82dab39f3a5e10d23420d1ebcd not found: ID does not exist"
Jul 10 23:02:13.230571 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.230555    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 10 23:02:13.233076 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.232732    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 10 23:02:13.234191 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.234168    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 23:02:13.328254 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.328209    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/df28b342-974c-4b66-95e1-15457303a2d7-config-out\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.328254 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.328255    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df28b342-974c-4b66-95e1-15457303a2d7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.328518 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.328318    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/df28b342-974c-4b66-95e1-15457303a2d7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.328518 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.328389    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/df28b342-974c-4b66-95e1-15457303a2d7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.328518 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.328423    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/df28b342-974c-4b66-95e1-15457303a2d7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.328518 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.328450    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/df28b342-974c-4b66-95e1-15457303a2d7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.328518 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.328469    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df28b342-974c-4b66-95e1-15457303a2d7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.328518 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.328491    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6p4c\" (UniqueName: \"kubernetes.io/projected/df28b342-974c-4b66-95e1-15457303a2d7-kube-api-access-x6p4c\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.328766 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.328533    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/df28b342-974c-4b66-95e1-15457303a2d7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.328766 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.328548    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df28b342-974c-4b66-95e1-15457303a2d7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.328766 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.328581    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/df28b342-974c-4b66-95e1-15457303a2d7-config\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.328766 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.328607    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/df28b342-974c-4b66-95e1-15457303a2d7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.328766 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.328626    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/df28b342-974c-4b66-95e1-15457303a2d7-web-config\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.328766 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.328651    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/df28b342-974c-4b66-95e1-15457303a2d7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.328766 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.328673    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/df28b342-974c-4b66-95e1-15457303a2d7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.328766 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.328692    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/df28b342-974c-4b66-95e1-15457303a2d7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.328766 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.328709    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/df28b342-974c-4b66-95e1-15457303a2d7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.328766 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.328731    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/df28b342-974c-4b66-95e1-15457303a2d7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.329094 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.328769    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/df28b342-974c-4b66-95e1-15457303a2d7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.329094 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.328792    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df28b342-974c-4b66-95e1-15457303a2d7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.429514 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.429422    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/df28b342-974c-4b66-95e1-15457303a2d7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.429514 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.429469    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/df28b342-974c-4b66-95e1-15457303a2d7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.429514 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.429489    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/df28b342-974c-4b66-95e1-15457303a2d7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.429725 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.429518    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df28b342-974c-4b66-95e1-15457303a2d7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.429725 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.429636    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6p4c\" (UniqueName: \"kubernetes.io/projected/df28b342-974c-4b66-95e1-15457303a2d7-kube-api-access-x6p4c\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.429725 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.429681    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/df28b342-974c-4b66-95e1-15457303a2d7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.429725 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.429705    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df28b342-974c-4b66-95e1-15457303a2d7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.429909 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.429735    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/df28b342-974c-4b66-95e1-15457303a2d7-config\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.429909 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.429764    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/df28b342-974c-4b66-95e1-15457303a2d7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.429909 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.429792    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/df28b342-974c-4b66-95e1-15457303a2d7-web-config\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.429909 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.429832    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/df28b342-974c-4b66-95e1-15457303a2d7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.429909 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.429858    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/df28b342-974c-4b66-95e1-15457303a2d7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.429909 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.429880    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/df28b342-974c-4b66-95e1-15457303a2d7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.429909 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.429908    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/df28b342-974c-4b66-95e1-15457303a2d7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.430244 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.429939    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/df28b342-974c-4b66-95e1-15457303a2d7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.430244 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.429967    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/df28b342-974c-4b66-95e1-15457303a2d7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.430244 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.429999    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df28b342-974c-4b66-95e1-15457303a2d7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.430244 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.430032    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/df28b342-974c-4b66-95e1-15457303a2d7-config-out\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.430244 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.430057    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df28b342-974c-4b66-95e1-15457303a2d7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.430244 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.430092    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/df28b342-974c-4b66-95e1-15457303a2d7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.430578 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.430272    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/df28b342-974c-4b66-95e1-15457303a2d7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.430871 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.430628    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df28b342-974c-4b66-95e1-15457303a2d7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.430871 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.430673    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/df28b342-974c-4b66-95e1-15457303a2d7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.431026 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.430965    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/df28b342-974c-4b66-95e1-15457303a2d7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.431087 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.431062    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df28b342-974c-4b66-95e1-15457303a2d7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.433128 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.432815    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/df28b342-974c-4b66-95e1-15457303a2d7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.433128 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.432998    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/df28b342-974c-4b66-95e1-15457303a2d7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.433128 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.433016    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/df28b342-974c-4b66-95e1-15457303a2d7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.433587 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.433444    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/df28b342-974c-4b66-95e1-15457303a2d7-web-config\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.433587 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.433511    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/df28b342-974c-4b66-95e1-15457303a2d7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.433732 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.433710    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/df28b342-974c-4b66-95e1-15457303a2d7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.434104 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.434044    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/df28b342-974c-4b66-95e1-15457303a2d7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.434104 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.434055    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/df28b342-974c-4b66-95e1-15457303a2d7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.434324 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.434280    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df28b342-974c-4b66-95e1-15457303a2d7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.434491 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.434471    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/df28b342-974c-4b66-95e1-15457303a2d7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.435815 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.435793    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/df28b342-974c-4b66-95e1-15457303a2d7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.436068 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.436048    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/df28b342-974c-4b66-95e1-15457303a2d7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.436152 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.436056    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/df28b342-974c-4b66-95e1-15457303a2d7-config-out\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.436265 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.436249    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/df28b342-974c-4b66-95e1-15457303a2d7-config\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.438717 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.438691    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6p4c\" (UniqueName: \"kubernetes.io/projected/df28b342-974c-4b66-95e1-15457303a2d7-kube-api-access-x6p4c\") pod \"prometheus-k8s-0\" (UID: \"df28b342-974c-4b66-95e1-15457303a2d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.538026 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.537969    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:13.559166 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.559128    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bddddf0-512e-466c-9327-b88418a330e4" path="/var/lib/kubelet/pods/2bddddf0-512e-466c-9327-b88418a330e4/volumes"
Jul 10 23:02:13.685256 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:13.685067    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 10 23:02:13.687652 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:02:13.687624    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf28b342_974c_4b66_95e1_15457303a2d7.slice/crio-270ec97fb17f02ff430bb56dbb6e07665071c351da56fd4b225c66750bfbb836 WatchSource:0}: Error finding container 270ec97fb17f02ff430bb56dbb6e07665071c351da56fd4b225c66750bfbb836: Status 404 returned error can't find the container with id 270ec97fb17f02ff430bb56dbb6e07665071c351da56fd4b225c66750bfbb836
Jul 10 23:02:14.168024 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:14.167992    2563 generic.go:356] "Generic (PLEG): container finished" podID="df28b342-974c-4b66-95e1-15457303a2d7" containerID="bb329e0854e1a6c63917a07a00d49648ed8424c1173546db7c60d131b95d5dc1" exitCode=0
Jul 10 23:02:14.168513 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:14.168077    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"df28b342-974c-4b66-95e1-15457303a2d7","Type":"ContainerDied","Data":"bb329e0854e1a6c63917a07a00d49648ed8424c1173546db7c60d131b95d5dc1"}
Jul 10 23:02:14.168513 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:14.168154    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"df28b342-974c-4b66-95e1-15457303a2d7","Type":"ContainerStarted","Data":"270ec97fb17f02ff430bb56dbb6e07665071c351da56fd4b225c66750bfbb836"}
Jul 10 23:02:15.035232 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:15.035199    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:02:15.087714 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:15.087680    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:02:15.178124 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:15.178083    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"df28b342-974c-4b66-95e1-15457303a2d7","Type":"ContainerStarted","Data":"03ee718c38928774c38465f6543e7e284e8f080bbd017119bf350bd47e5f9dfb"}
Jul 10 23:02:15.178540 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:15.178170    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"df28b342-974c-4b66-95e1-15457303a2d7","Type":"ContainerStarted","Data":"ea61b19a4518b307a9fee67bda6cc795e9410a9a94f241c461efc1da221b3f70"}
Jul 10 23:02:15.178540 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:15.178217    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"df28b342-974c-4b66-95e1-15457303a2d7","Type":"ContainerStarted","Data":"158c3d50dd671a56db9e790c2bab0f8fa7c3f608787f2e7ac8fc4aa90f802a1f"}
Jul 10 23:02:15.178540 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:15.178262    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"df28b342-974c-4b66-95e1-15457303a2d7","Type":"ContainerStarted","Data":"a267a69b61708a405801fa87b46c74fcc54a29383f1c94e69967cb5f6edff077"}
Jul 10 23:02:15.178540 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:15.178297    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"df28b342-974c-4b66-95e1-15457303a2d7","Type":"ContainerStarted","Data":"0d918e1b91fecd34dfb6e9f453e193e281420d0f3b9bb8135a166a8b21bbef90"}
Jul 10 23:02:15.178540 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:15.178332    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"df28b342-974c-4b66-95e1-15457303a2d7","Type":"ContainerStarted","Data":"228543a351b5a1923c9953db624009a1b7e54a344548e4cbe14128252b49eb6f"}
Jul 10 23:02:15.211600 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:15.209761    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.188978434 podStartE2EDuration="2.20973574s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.757306ms" isStatefulPod=false podCreationTimestamp="2026-07-10 23:02:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:02:15.20335073 +0000 UTC m=+396.090086886" watchObservedRunningTime="2026-07-10 23:02:15.20973574 +0000 UTC m=+396.096471913"
Jul 10 23:02:18.538392 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:18.538327    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:02:18.937240 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:18.937132    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:02:19.041077 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:19.041024    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:02:19.095892 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:19.095864    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:02:26.238007 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:26.237972    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:02:26.336145 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:26.336100    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:02:37.518186 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:37.518140    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:02:52.223688 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:52.223630    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:02:52.316387 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:52.316311    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:02:52.744937 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:02:52.744900    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:03:12.984395 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:12.984241    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:03:13.085039 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:13.084995    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:03:13.538229 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:13.538174    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:03:13.555407 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:13.555376    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:03:14.445077 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:14.445049    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 10 23:03:18.739396 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:18.739309    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:03:20.137404 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:20.137320    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:03:22.338698 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:22.338653    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:03:22.439501 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:22.439467    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:03:24.029051 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:24.029014    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:03:24.153375 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:24.153338    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:03:24.588459 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:24.588423    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:03:24.686519 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:24.686486    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:03:26.793301 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:26.793268    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:03:27.032632 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:27.032604    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:03:29.335598 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:29.335560    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:03:29.393517 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:29.393477    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:03:32.925804 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:32.925754    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:03:33.025693 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:33.025643    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:03:34.997608 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:34.997568    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:03:45.114649 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:45.114607    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-dcw54"]
Jul 10 23:03:45.117954 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:45.117928    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-dcw54"
Jul 10 23:03:45.120473 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:45.120452    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 10 23:03:45.120594 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:45.120521    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 10 23:03:45.121145 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:45.121128    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-g9rtx\""
Jul 10 23:03:45.127928 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:45.127903    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-dcw54"]
Jul 10 23:03:45.235839 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:45.235798    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwft6\" (UniqueName: \"kubernetes.io/projected/304301ab-f1a8-4b90-8593-3b5c2d507fbc-kube-api-access-cwft6\") pod \"cert-manager-55b4d954d5-dcw54\" (UID: \"304301ab-f1a8-4b90-8593-3b5c2d507fbc\") " pod="cert-manager/cert-manager-55b4d954d5-dcw54"
Jul 10 23:03:45.236028 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:45.235890    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/304301ab-f1a8-4b90-8593-3b5c2d507fbc-bound-sa-token\") pod \"cert-manager-55b4d954d5-dcw54\" (UID: \"304301ab-f1a8-4b90-8593-3b5c2d507fbc\") " pod="cert-manager/cert-manager-55b4d954d5-dcw54"
Jul 10 23:03:45.337228 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:45.337195    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwft6\" (UniqueName: \"kubernetes.io/projected/304301ab-f1a8-4b90-8593-3b5c2d507fbc-kube-api-access-cwft6\") pod \"cert-manager-55b4d954d5-dcw54\" (UID: \"304301ab-f1a8-4b90-8593-3b5c2d507fbc\") " pod="cert-manager/cert-manager-55b4d954d5-dcw54"
Jul 10 23:03:45.337439 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:45.337275    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/304301ab-f1a8-4b90-8593-3b5c2d507fbc-bound-sa-token\") pod \"cert-manager-55b4d954d5-dcw54\" (UID: \"304301ab-f1a8-4b90-8593-3b5c2d507fbc\") " pod="cert-manager/cert-manager-55b4d954d5-dcw54"
Jul 10 23:03:45.346085 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:45.346043    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/304301ab-f1a8-4b90-8593-3b5c2d507fbc-bound-sa-token\") pod \"cert-manager-55b4d954d5-dcw54\" (UID: \"304301ab-f1a8-4b90-8593-3b5c2d507fbc\") " pod="cert-manager/cert-manager-55b4d954d5-dcw54"
Jul 10 23:03:45.346234 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:45.346167    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwft6\" (UniqueName: \"kubernetes.io/projected/304301ab-f1a8-4b90-8593-3b5c2d507fbc-kube-api-access-cwft6\") pod \"cert-manager-55b4d954d5-dcw54\" (UID: \"304301ab-f1a8-4b90-8593-3b5c2d507fbc\") " pod="cert-manager/cert-manager-55b4d954d5-dcw54"
Jul 10 23:03:45.442265 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:45.442162    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-dcw54"
Jul 10 23:03:45.583759 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:45.583720    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-dcw54"]
Jul 10 23:03:45.587049 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:03:45.587022    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod304301ab_f1a8_4b90_8593_3b5c2d507fbc.slice/crio-ffc3eba274bd6d574a023a63a87dc56aac76db010a77ea45e632978cc3965ee1 WatchSource:0}: Error finding container ffc3eba274bd6d574a023a63a87dc56aac76db010a77ea45e632978cc3965ee1: Status 404 returned error can't find the container with id ffc3eba274bd6d574a023a63a87dc56aac76db010a77ea45e632978cc3965ee1
Jul 10 23:03:46.563685 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:46.563615    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-dcw54" event={"ID":"304301ab-f1a8-4b90-8593-3b5c2d507fbc","Type":"ContainerStarted","Data":"ffc3eba274bd6d574a023a63a87dc56aac76db010a77ea45e632978cc3965ee1"}
Jul 10 23:03:47.080677 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:47.080620    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:03:47.178107 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:47.178065    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:03:48.575194 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:48.575160    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-dcw54" event={"ID":"304301ab-f1a8-4b90-8593-3b5c2d507fbc","Type":"ContainerStarted","Data":"be7d7fc19866d1b36d4c175460653b2958a71741ada1de0bb88c32cd286f4638"}
Jul 10 23:03:48.591440 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:48.591377    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-dcw54" podStartSLOduration=0.724396756 podStartE2EDuration="3.591335637s" totalImagesPullingTime="2.866938881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:03:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:03:48.590725079 +0000 UTC m=+489.477461247" watchObservedRunningTime="2026-07-10 23:03:48.591335637 +0000 UTC m=+489.478071781"
Jul 10 23:03:49.068726 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:49.068684    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2n95f"]
Jul 10 23:03:49.072173 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:49.072135    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2n95f"
Jul 10 23:03:49.074944 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:49.074913    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 10 23:03:49.075092 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:49.074913    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-sqvf7\""
Jul 10 23:03:49.075092 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:49.074919    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 10 23:03:49.082640 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:49.082613    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2n95f"]
Jul 10 23:03:49.167772 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:49.167735    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl8cb\" (UniqueName: \"kubernetes.io/projected/65f11176-0b83-48c0-ac77-fd3bd43ac0df-kube-api-access-nl8cb\") pod \"openshift-lws-operator-fd8ccff4c-2n95f\" (UID: \"65f11176-0b83-48c0-ac77-fd3bd43ac0df\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2n95f"
Jul 10 23:03:49.167949 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:49.167785    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65f11176-0b83-48c0-ac77-fd3bd43ac0df-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2n95f\" (UID: \"65f11176-0b83-48c0-ac77-fd3bd43ac0df\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2n95f"
Jul 10 23:03:49.268414 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:49.268341    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl8cb\" (UniqueName: \"kubernetes.io/projected/65f11176-0b83-48c0-ac77-fd3bd43ac0df-kube-api-access-nl8cb\") pod \"openshift-lws-operator-fd8ccff4c-2n95f\" (UID: \"65f11176-0b83-48c0-ac77-fd3bd43ac0df\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2n95f"
Jul 10 23:03:49.268588 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:49.268424    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65f11176-0b83-48c0-ac77-fd3bd43ac0df-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2n95f\" (UID: \"65f11176-0b83-48c0-ac77-fd3bd43ac0df\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2n95f"
Jul 10 23:03:49.268751 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:49.268735    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65f11176-0b83-48c0-ac77-fd3bd43ac0df-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2n95f\" (UID: \"65f11176-0b83-48c0-ac77-fd3bd43ac0df\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2n95f"
Jul 10 23:03:49.280705 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:49.280674    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl8cb\" (UniqueName: \"kubernetes.io/projected/65f11176-0b83-48c0-ac77-fd3bd43ac0df-kube-api-access-nl8cb\") pod \"openshift-lws-operator-fd8ccff4c-2n95f\" (UID: \"65f11176-0b83-48c0-ac77-fd3bd43ac0df\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2n95f"
Jul 10 23:03:49.381825 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:49.381737    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2n95f"
Jul 10 23:03:49.522391 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:49.522230    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2n95f"]
Jul 10 23:03:49.524873 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:03:49.524840    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65f11176_0b83_48c0_ac77_fd3bd43ac0df.slice/crio-8a5bd0f4a2f0b7f9f1e7b154ece73da0882ad528a32ddede597af5684834d929 WatchSource:0}: Error finding container 8a5bd0f4a2f0b7f9f1e7b154ece73da0882ad528a32ddede597af5684834d929: Status 404 returned error can't find the container with id 8a5bd0f4a2f0b7f9f1e7b154ece73da0882ad528a32ddede597af5684834d929
Jul 10 23:03:49.582958 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:49.582924    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2n95f" event={"ID":"65f11176-0b83-48c0-ac77-fd3bd43ac0df","Type":"ContainerStarted","Data":"8a5bd0f4a2f0b7f9f1e7b154ece73da0882ad528a32ddede597af5684834d929"}
Jul 10 23:03:52.599467 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:52.599421    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2n95f" event={"ID":"65f11176-0b83-48c0-ac77-fd3bd43ac0df","Type":"ContainerStarted","Data":"6a83ad2081faaae71dcf1d56414ee65faf830a16b5676931fb9b307cc6643ffe"}
Jul 10 23:03:52.614420 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:52.614345    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2n95f" podStartSLOduration=1.054584235 podStartE2EDuration="3.61432546s" totalImagesPullingTime="2.559741225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:03:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:03:52.613496519 +0000 UTC m=+493.500232675" watchObservedRunningTime="2026-07-10 23:03:52.61432546 +0000 UTC m=+493.501061605"
Jul 10 23:03:53.289680 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:53.289638    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:03:53.390343 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:53.390302    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:03:55.212088 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:55.212030    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:03:59.483498 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:59.483458    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:03:59.519743 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:59.519710    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-85bcb67fdb-224bc"]
Jul 10 23:03:59.522866 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:59.522841    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-85bcb67fdb-224bc"
Jul 10 23:03:59.526083 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:59.526048    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 10 23:03:59.526220 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:59.526087    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 10 23:03:59.526220 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:59.526054    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-n9bdm\""
Jul 10 23:03:59.526220 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:59.526129    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 10 23:03:59.534680 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:59.534654    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-85bcb67fdb-224bc"]
Jul 10 23:03:59.553490 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:59.553457    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ad95df28-76f7-4c62-ac24-858d7303eaed-manager-config\") pod \"lws-controller-manager-85bcb67fdb-224bc\" (UID: \"ad95df28-76f7-4c62-ac24-858d7303eaed\") " pod="openshift-lws-operator/lws-controller-manager-85bcb67fdb-224bc"
Jul 10 23:03:59.553667 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:59.553496    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ad95df28-76f7-4c62-ac24-858d7303eaed-metrics-cert\") pod \"lws-controller-manager-85bcb67fdb-224bc\" (UID: \"ad95df28-76f7-4c62-ac24-858d7303eaed\") " pod="openshift-lws-operator/lws-controller-manager-85bcb67fdb-224bc"
Jul 10 23:03:59.553667 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:59.553516    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7d8c\" (UniqueName: \"kubernetes.io/projected/ad95df28-76f7-4c62-ac24-858d7303eaed-kube-api-access-t7d8c\") pod \"lws-controller-manager-85bcb67fdb-224bc\" (UID: \"ad95df28-76f7-4c62-ac24-858d7303eaed\") " pod="openshift-lws-operator/lws-controller-manager-85bcb67fdb-224bc"
Jul 10 23:03:59.553667 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:59.553643    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad95df28-76f7-4c62-ac24-858d7303eaed-cert\") pod \"lws-controller-manager-85bcb67fdb-224bc\" (UID: \"ad95df28-76f7-4c62-ac24-858d7303eaed\") " pod="openshift-lws-operator/lws-controller-manager-85bcb67fdb-224bc"
Jul 10 23:03:59.582624 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:59.582590    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:03:59.655002 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:59.654968    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ad95df28-76f7-4c62-ac24-858d7303eaed-manager-config\") pod \"lws-controller-manager-85bcb67fdb-224bc\" (UID: \"ad95df28-76f7-4c62-ac24-858d7303eaed\") " pod="openshift-lws-operator/lws-controller-manager-85bcb67fdb-224bc"
Jul 10 23:03:59.655176 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:59.655009    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ad95df28-76f7-4c62-ac24-858d7303eaed-metrics-cert\") pod \"lws-controller-manager-85bcb67fdb-224bc\" (UID: \"ad95df28-76f7-4c62-ac24-858d7303eaed\") " pod="openshift-lws-operator/lws-controller-manager-85bcb67fdb-224bc"
Jul 10 23:03:59.655176 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:59.655029    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7d8c\" (UniqueName: \"kubernetes.io/projected/ad95df28-76f7-4c62-ac24-858d7303eaed-kube-api-access-t7d8c\") pod \"lws-controller-manager-85bcb67fdb-224bc\" (UID: \"ad95df28-76f7-4c62-ac24-858d7303eaed\") " pod="openshift-lws-operator/lws-controller-manager-85bcb67fdb-224bc"
Jul 10 23:03:59.655176 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:59.655078    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad95df28-76f7-4c62-ac24-858d7303eaed-cert\") pod \"lws-controller-manager-85bcb67fdb-224bc\" (UID: \"ad95df28-76f7-4c62-ac24-858d7303eaed\") " pod="openshift-lws-operator/lws-controller-manager-85bcb67fdb-224bc"
Jul 10 23:03:59.655800 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:59.655776    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ad95df28-76f7-4c62-ac24-858d7303eaed-manager-config\") pod \"lws-controller-manager-85bcb67fdb-224bc\" (UID: \"ad95df28-76f7-4c62-ac24-858d7303eaed\") " pod="openshift-lws-operator/lws-controller-manager-85bcb67fdb-224bc"
Jul 10 23:03:59.657832 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:59.657803    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad95df28-76f7-4c62-ac24-858d7303eaed-cert\") pod \"lws-controller-manager-85bcb67fdb-224bc\" (UID: \"ad95df28-76f7-4c62-ac24-858d7303eaed\") " pod="openshift-lws-operator/lws-controller-manager-85bcb67fdb-224bc"
Jul 10 23:03:59.657935 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:59.657837    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ad95df28-76f7-4c62-ac24-858d7303eaed-metrics-cert\") pod \"lws-controller-manager-85bcb67fdb-224bc\" (UID: \"ad95df28-76f7-4c62-ac24-858d7303eaed\") " pod="openshift-lws-operator/lws-controller-manager-85bcb67fdb-224bc"
Jul 10 23:03:59.665336 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:59.664750    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7d8c\" (UniqueName: \"kubernetes.io/projected/ad95df28-76f7-4c62-ac24-858d7303eaed-kube-api-access-t7d8c\") pod \"lws-controller-manager-85bcb67fdb-224bc\" (UID: \"ad95df28-76f7-4c62-ac24-858d7303eaed\") " pod="openshift-lws-operator/lws-controller-manager-85bcb67fdb-224bc"
Jul 10 23:03:59.832786 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:59.832670    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-85bcb67fdb-224bc"
Jul 10 23:03:59.975556 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:03:59.975518    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-85bcb67fdb-224bc"]
Jul 10 23:03:59.978630 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:03:59.978599    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad95df28_76f7_4c62_ac24_858d7303eaed.slice/crio-9829394f8a048e779cd0459aae8629c065cb47dc83b1947e0c12acbe8ff78728 WatchSource:0}: Error finding container 9829394f8a048e779cd0459aae8629c065cb47dc83b1947e0c12acbe8ff78728: Status 404 returned error can't find the container with id 9829394f8a048e779cd0459aae8629c065cb47dc83b1947e0c12acbe8ff78728
Jul 10 23:04:00.634233 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:00.634200    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-85bcb67fdb-224bc" event={"ID":"ad95df28-76f7-4c62-ac24-858d7303eaed","Type":"ContainerStarted","Data":"9829394f8a048e779cd0459aae8629c065cb47dc83b1947e0c12acbe8ff78728"}
Jul 10 23:04:02.645065 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:02.645023    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-85bcb67fdb-224bc" event={"ID":"ad95df28-76f7-4c62-ac24-858d7303eaed","Type":"ContainerStarted","Data":"a7910d86de6a43de2b3fc85bd6ce49e1780467d9978d80b3982687e58bc8c5d6"}
Jul 10 23:04:02.645471 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:02.645128    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-85bcb67fdb-224bc"
Jul 10 23:04:02.661626 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:02.661575    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-85bcb67fdb-224bc" podStartSLOduration=1.919680995 podStartE2EDuration="3.661557319s" totalImagesPullingTime="1.741876324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:03:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:04:02.659634248 +0000 UTC m=+503.546370390" watchObservedRunningTime="2026-07-10 23:04:02.661557319 +0000 UTC m=+503.548293541"
Jul 10 23:04:08.217477 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:08.217438    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:08.310489 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:08.310452    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:08.824992 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:08.824959    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:09.057670 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.057632    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:09.160568 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.160487    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:09.209557 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.209523    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58bf55498c-lxs9p"]
Jul 10 23:04:09.216491 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.216465    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58bf55498c-lxs9p"
Jul 10 23:04:09.220217 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.220189    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-94qk9\""
Jul 10 23:04:09.220641 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.220248    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 10 23:04:09.220641 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.220576    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 10 23:04:09.220866 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.220852    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 10 23:04:09.221327 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.221312    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 10 23:04:09.239144 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.239109    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58bf55498c-lxs9p"]
Jul 10 23:04:09.341576 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.341534    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:09.341804 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.341788    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/96854ace-3322-4255-94bd-eb446df0a254-tmp\") pod \"opendatahub-operator-controller-manager-58bf55498c-lxs9p\" (UID: \"96854ace-3322-4255-94bd-eb446df0a254\") " pod="opendatahub/opendatahub-operator-controller-manager-58bf55498c-lxs9p"
Jul 10 23:04:09.341850 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.341814    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wp8z\" (UniqueName: \"kubernetes.io/projected/96854ace-3322-4255-94bd-eb446df0a254-kube-api-access-8wp8z\") pod \"opendatahub-operator-controller-manager-58bf55498c-lxs9p\" (UID: \"96854ace-3322-4255-94bd-eb446df0a254\") " pod="opendatahub/opendatahub-operator-controller-manager-58bf55498c-lxs9p"
Jul 10 23:04:09.342036 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.341854    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/96854ace-3322-4255-94bd-eb446df0a254-manifests\") pod \"opendatahub-operator-controller-manager-58bf55498c-lxs9p\" (UID: \"96854ace-3322-4255-94bd-eb446df0a254\") " pod="opendatahub/opendatahub-operator-controller-manager-58bf55498c-lxs9p"
Jul 10 23:04:09.342036 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.341919    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/96854ace-3322-4255-94bd-eb446df0a254-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58bf55498c-lxs9p\" (UID: \"96854ace-3322-4255-94bd-eb446df0a254\") " pod="opendatahub/opendatahub-operator-controller-manager-58bf55498c-lxs9p"
Jul 10 23:04:09.342036 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.341962    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/96854ace-3322-4255-94bd-eb446df0a254-webhook-cert\") pod \"opendatahub-operator-controller-manager-58bf55498c-lxs9p\" (UID: \"96854ace-3322-4255-94bd-eb446df0a254\") " pod="opendatahub/opendatahub-operator-controller-manager-58bf55498c-lxs9p"
Jul 10 23:04:09.442686 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.442588    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/96854ace-3322-4255-94bd-eb446df0a254-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58bf55498c-lxs9p\" (UID: \"96854ace-3322-4255-94bd-eb446df0a254\") " pod="opendatahub/opendatahub-operator-controller-manager-58bf55498c-lxs9p"
Jul 10 23:04:09.442686 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.442631    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/96854ace-3322-4255-94bd-eb446df0a254-webhook-cert\") pod \"opendatahub-operator-controller-manager-58bf55498c-lxs9p\" (UID: \"96854ace-3322-4255-94bd-eb446df0a254\") " pod="opendatahub/opendatahub-operator-controller-manager-58bf55498c-lxs9p"
Jul 10 23:04:09.442686 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.442686    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/96854ace-3322-4255-94bd-eb446df0a254-tmp\") pod \"opendatahub-operator-controller-manager-58bf55498c-lxs9p\" (UID: \"96854ace-3322-4255-94bd-eb446df0a254\") " pod="opendatahub/opendatahub-operator-controller-manager-58bf55498c-lxs9p"
Jul 10 23:04:09.442952 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.442703    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wp8z\" (UniqueName: \"kubernetes.io/projected/96854ace-3322-4255-94bd-eb446df0a254-kube-api-access-8wp8z\") pod \"opendatahub-operator-controller-manager-58bf55498c-lxs9p\" (UID: \"96854ace-3322-4255-94bd-eb446df0a254\") " pod="opendatahub/opendatahub-operator-controller-manager-58bf55498c-lxs9p"
Jul 10 23:04:09.442952 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.442738    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/96854ace-3322-4255-94bd-eb446df0a254-manifests\") pod \"opendatahub-operator-controller-manager-58bf55498c-lxs9p\" (UID: \"96854ace-3322-4255-94bd-eb446df0a254\") " pod="opendatahub/opendatahub-operator-controller-manager-58bf55498c-lxs9p"
Jul 10 23:04:09.443170 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.443146    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/96854ace-3322-4255-94bd-eb446df0a254-manifests\") pod \"opendatahub-operator-controller-manager-58bf55498c-lxs9p\" (UID: \"96854ace-3322-4255-94bd-eb446df0a254\") " pod="opendatahub/opendatahub-operator-controller-manager-58bf55498c-lxs9p"
Jul 10 23:04:09.445201 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.445169    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/96854ace-3322-4255-94bd-eb446df0a254-tmp\") pod \"opendatahub-operator-controller-manager-58bf55498c-lxs9p\" (UID: \"96854ace-3322-4255-94bd-eb446df0a254\") " pod="opendatahub/opendatahub-operator-controller-manager-58bf55498c-lxs9p"
Jul 10 23:04:09.445338 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.445244    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/96854ace-3322-4255-94bd-eb446df0a254-webhook-cert\") pod \"opendatahub-operator-controller-manager-58bf55498c-lxs9p\" (UID: \"96854ace-3322-4255-94bd-eb446df0a254\") " pod="opendatahub/opendatahub-operator-controller-manager-58bf55498c-lxs9p"
Jul 10 23:04:09.445338 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.445287    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/96854ace-3322-4255-94bd-eb446df0a254-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58bf55498c-lxs9p\" (UID: \"96854ace-3322-4255-94bd-eb446df0a254\") " pod="opendatahub/opendatahub-operator-controller-manager-58bf55498c-lxs9p"
Jul 10 23:04:09.448791 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.448761    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:09.451016 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.450997    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wp8z\" (UniqueName: \"kubernetes.io/projected/96854ace-3322-4255-94bd-eb446df0a254-kube-api-access-8wp8z\") pod \"opendatahub-operator-controller-manager-58bf55498c-lxs9p\" (UID: \"96854ace-3322-4255-94bd-eb446df0a254\") " pod="opendatahub/opendatahub-operator-controller-manager-58bf55498c-lxs9p"
Jul 10 23:04:09.527865 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.527828    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58bf55498c-lxs9p"
Jul 10 23:04:09.549737 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.549702    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:09.638806 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.638769    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:09.674962 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.674933    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58bf55498c-lxs9p"]
Jul 10 23:04:09.679163 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:04:09.679125    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96854ace_3322_4255_94bd_eb446df0a254.slice/crio-a140fea2b0b815effd05078b431c45a3da0b7d222be915ca9cb9503d659f086b WatchSource:0}: Error finding container a140fea2b0b815effd05078b431c45a3da0b7d222be915ca9cb9503d659f086b: Status 404 returned error can't find the container with id a140fea2b0b815effd05078b431c45a3da0b7d222be915ca9cb9503d659f086b
Jul 10 23:04:09.707884 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:09.707787    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:10.691946 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:10.691899    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58bf55498c-lxs9p" event={"ID":"96854ace-3322-4255-94bd-eb446df0a254","Type":"ContainerStarted","Data":"a140fea2b0b815effd05078b431c45a3da0b7d222be915ca9cb9503d659f086b"}
Jul 10 23:04:12.703296 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:12.703264    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58bf55498c-lxs9p" event={"ID":"96854ace-3322-4255-94bd-eb446df0a254","Type":"ContainerStarted","Data":"0259d4224389dd4a1b344b89416101787fc0b21c1137ca0a6cd6d6dbc317f87b"}
Jul 10 23:04:13.652459 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:13.652429    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-85bcb67fdb-224bc"
Jul 10 23:04:14.092283 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:14.092242    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:14.193483 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:14.193447    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:14.713667 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:14.713632    2563 generic.go:356] "Generic (PLEG): container finished" podID="96854ace-3322-4255-94bd-eb446df0a254" containerID="0259d4224389dd4a1b344b89416101787fc0b21c1137ca0a6cd6d6dbc317f87b" exitCode=0
Jul 10 23:04:14.713863 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:14.713709    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58bf55498c-lxs9p" event={"ID":"96854ace-3322-4255-94bd-eb446df0a254","Type":"ContainerDied","Data":"0259d4224389dd4a1b344b89416101787fc0b21c1137ca0a6cd6d6dbc317f87b"}
Jul 10 23:04:15.720167 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:15.720133    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58bf55498c-lxs9p" event={"ID":"96854ace-3322-4255-94bd-eb446df0a254","Type":"ContainerStarted","Data":"40c32455409ecbc953551fc23da714bc5fbaf2e791330499a039189a70c155f3"}
Jul 10 23:04:15.720562 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:15.720346    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-58bf55498c-lxs9p"
Jul 10 23:04:15.741242 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:15.741181    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-58bf55498c-lxs9p" podStartSLOduration=2.261319348 podStartE2EDuration="6.741162232s" totalImagesPullingTime="3.321282617s" totalInitContainerRuntime="1.158560267s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:04:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:04:15.739298157 +0000 UTC m=+516.626034325" watchObservedRunningTime="2026-07-10 23:04:15.741162232 +0000 UTC m=+516.627898375"
Jul 10 23:04:26.727740 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:26.727708    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-58bf55498c-lxs9p"
Jul 10 23:04:28.388788 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:28.388751    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:28.518239 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:28.518205    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:28.929226 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:28.929193    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:29.021831 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:29.021799    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7947c57c45-c46r7"]
Jul 10 23:04:29.030386 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:29.030341    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7947c57c45-c46r7"
Jul 10 23:04:29.034589 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:29.034561    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 10 23:04:29.034589 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:29.034571    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 10 23:04:29.034833 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:29.034752    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 10 23:04:29.034995 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:29.034980    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 10 23:04:29.035739 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:29.035721    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-vg7gz\""
Jul 10 23:04:29.049843 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:29.049810    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7947c57c45-c46r7"]
Jul 10 23:04:29.124714 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:29.124671    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98989cf4-283f-4a9d-82b0-9b4f967d39c9-tmp\") pod \"kube-auth-proxy-7947c57c45-c46r7\" (UID: \"98989cf4-283f-4a9d-82b0-9b4f967d39c9\") " pod="openshift-ingress/kube-auth-proxy-7947c57c45-c46r7"
Jul 10 23:04:29.124916 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:29.124784    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f675\" (UniqueName: \"kubernetes.io/projected/98989cf4-283f-4a9d-82b0-9b4f967d39c9-kube-api-access-2f675\") pod \"kube-auth-proxy-7947c57c45-c46r7\" (UID: \"98989cf4-283f-4a9d-82b0-9b4f967d39c9\") " pod="openshift-ingress/kube-auth-proxy-7947c57c45-c46r7"
Jul 10 23:04:29.124916 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:29.124820    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98989cf4-283f-4a9d-82b0-9b4f967d39c9-tls-certs\") pod \"kube-auth-proxy-7947c57c45-c46r7\" (UID: \"98989cf4-283f-4a9d-82b0-9b4f967d39c9\") " pod="openshift-ingress/kube-auth-proxy-7947c57c45-c46r7"
Jul 10 23:04:29.198778 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:29.198677    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:29.225942 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:29.225903    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f675\" (UniqueName: \"kubernetes.io/projected/98989cf4-283f-4a9d-82b0-9b4f967d39c9-kube-api-access-2f675\") pod \"kube-auth-proxy-7947c57c45-c46r7\" (UID: \"98989cf4-283f-4a9d-82b0-9b4f967d39c9\") " pod="openshift-ingress/kube-auth-proxy-7947c57c45-c46r7"
Jul 10 23:04:29.225942 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:29.225946    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98989cf4-283f-4a9d-82b0-9b4f967d39c9-tls-certs\") pod \"kube-auth-proxy-7947c57c45-c46r7\" (UID: \"98989cf4-283f-4a9d-82b0-9b4f967d39c9\") " pod="openshift-ingress/kube-auth-proxy-7947c57c45-c46r7"
Jul 10 23:04:29.226207 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:29.225996    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98989cf4-283f-4a9d-82b0-9b4f967d39c9-tmp\") pod \"kube-auth-proxy-7947c57c45-c46r7\" (UID: \"98989cf4-283f-4a9d-82b0-9b4f967d39c9\") " pod="openshift-ingress/kube-auth-proxy-7947c57c45-c46r7"
Jul 10 23:04:29.228966 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:29.228932    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98989cf4-283f-4a9d-82b0-9b4f967d39c9-tmp\") pod \"kube-auth-proxy-7947c57c45-c46r7\" (UID: \"98989cf4-283f-4a9d-82b0-9b4f967d39c9\") " pod="openshift-ingress/kube-auth-proxy-7947c57c45-c46r7"
Jul 10 23:04:29.229194 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:29.229175    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98989cf4-283f-4a9d-82b0-9b4f967d39c9-tls-certs\") pod \"kube-auth-proxy-7947c57c45-c46r7\" (UID: \"98989cf4-283f-4a9d-82b0-9b4f967d39c9\") " pod="openshift-ingress/kube-auth-proxy-7947c57c45-c46r7"
Jul 10 23:04:29.238320 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:29.238287    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f675\" (UniqueName: \"kubernetes.io/projected/98989cf4-283f-4a9d-82b0-9b4f967d39c9-kube-api-access-2f675\") pod \"kube-auth-proxy-7947c57c45-c46r7\" (UID: \"98989cf4-283f-4a9d-82b0-9b4f967d39c9\") " pod="openshift-ingress/kube-auth-proxy-7947c57c45-c46r7"
Jul 10 23:04:29.286896 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:29.286859    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:29.339860 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:29.339824    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7947c57c45-c46r7"
Jul 10 23:04:29.386338 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:29.386307    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:29.493193 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:29.493160    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:29.506346 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:29.506318    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7947c57c45-c46r7"]
Jul 10 23:04:29.508998 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:04:29.508970    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98989cf4_283f_4a9d_82b0_9b4f967d39c9.slice/crio-1be436680159722652833a47c01a6101f564899812a24281aaa7b33218fb9d6b WatchSource:0}: Error finding container 1be436680159722652833a47c01a6101f564899812a24281aaa7b33218fb9d6b: Status 404 returned error can't find the container with id 1be436680159722652833a47c01a6101f564899812a24281aaa7b33218fb9d6b
Jul 10 23:04:29.786856 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:29.786751    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7947c57c45-c46r7" event={"ID":"98989cf4-283f-4a9d-82b0-9b4f967d39c9","Type":"ContainerStarted","Data":"1be436680159722652833a47c01a6101f564899812a24281aaa7b33218fb9d6b"}
Jul 10 23:04:29.787497 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:29.787473    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:30.163631 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:30.163543    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:30.253040 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:30.252983    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:30.395692 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:30.395660    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:30.467268 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:30.467150    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:31.388632 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:31.388598    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:32.801326 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:32.801287    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7947c57c45-c46r7" event={"ID":"98989cf4-283f-4a9d-82b0-9b4f967d39c9","Type":"ContainerStarted","Data":"006ab25434e6dcf7213c33f4721766bb190ccdb7d6d46ec5cc6f2410569326f4"}
Jul 10 23:04:32.818801 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:32.818746    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7947c57c45-c46r7" podStartSLOduration=1.753367481 podStartE2EDuration="4.818729328s" totalImagesPullingTime="3.065361847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:04:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:04:32.818405728 +0000 UTC m=+533.705141868" watchObservedRunningTime="2026-07-10 23:04:32.818729328 +0000 UTC m=+533.705465470"
Jul 10 23:04:36.235800 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:36.235764    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:36.402146 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:36.402109    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:37.021094 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:37.021051    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:37.232691 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:37.232652    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:37.332189 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:37.332096    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:39.187112 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:39.187070    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:39.289698 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:39.289661    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:39.417827 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:39.417783    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:39.626493 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:39.626451    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:40.552977 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:40.552930    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:40.635744 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:40.635686    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:40.736182 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:40.736147    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:41.252928 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:41.252890    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:47.573536 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:47.573496    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:04:58.474969 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:04:58.474911    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:05:25.047405 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:05:25.047329    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:05:26.995874 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:05:26.995845    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:05:27.096200 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:05:27.096169    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:05:29.845893 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:05:29.845854    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:05:29.949159 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:05:29.949126    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:05:30.016835 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:05:30.016809    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:05:30.772846 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:05:30.772804    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:05:30.985516 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:05:30.985475    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:05:39.466320 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:05:39.466287    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/ovn-acl-logging/0.log"
Jul 10 23:05:39.466768 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:05:39.466744    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/ovn-acl-logging/0.log"
Jul 10 23:05:40.826308 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:05:40.826267    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:05:40.878696 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:05:40.878663    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:05:58.513594 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:05:58.513550    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:05:59.599857 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:05:59.599807    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:04.116158 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:04.116123    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:05.958168 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:05.958133    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:08.594674 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:08.594642    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:09.325150 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:09.325115    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:10.358194 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:10.358145    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:11.928356 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:11.928320    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:12.548897 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:12.548862    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:14.022681 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:14.022643    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:14.119468 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:14.119430    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:14.926945 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:14.926911    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:14.991470 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:14.991434    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:15.322102 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:15.322069    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:15.424485 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:15.424449    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:17.804901 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:17.804857    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:18.114660 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:18.114555    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:19.326927 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:19.326894    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:19.387545 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:19.387505    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:25.066964 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:25.066926    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:25.120336 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:25.120307    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:25.182811 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:25.182774    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:26.151163 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:26.151123    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:26.255416 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:26.255354    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:28.630380 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:28.630317    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:28.734523 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:28.734474    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:28.939391 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:28.939289    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xr96m"]
Jul 10 23:06:28.943010 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:28.942989    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xr96m"
Jul 10 23:06:28.945790 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:28.945754    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-27xch\""
Jul 10 23:06:28.945939 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:28.945812    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 10 23:06:28.946612 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:28.946591    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 10 23:06:28.956325 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:28.956293    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xr96m"]
Jul 10 23:06:28.992193 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:28.992154    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqp2w\" (UniqueName: \"kubernetes.io/projected/09245303-0104-4b38-86ed-caa866cbf92d-kube-api-access-qqp2w\") pod \"limitador-operator-controller-manager-859d7b78bd-xr96m\" (UID: \"09245303-0104-4b38-86ed-caa866cbf92d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xr96m"
Jul 10 23:06:29.093517 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:29.093477    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqp2w\" (UniqueName: \"kubernetes.io/projected/09245303-0104-4b38-86ed-caa866cbf92d-kube-api-access-qqp2w\") pod \"limitador-operator-controller-manager-859d7b78bd-xr96m\" (UID: \"09245303-0104-4b38-86ed-caa866cbf92d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xr96m"
Jul 10 23:06:29.103634 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:29.103602    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqp2w\" (UniqueName: \"kubernetes.io/projected/09245303-0104-4b38-86ed-caa866cbf92d-kube-api-access-qqp2w\") pod \"limitador-operator-controller-manager-859d7b78bd-xr96m\" (UID: \"09245303-0104-4b38-86ed-caa866cbf92d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xr96m"
Jul 10 23:06:29.254933 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:29.254895    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xr96m"
Jul 10 23:06:29.410230 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:29.410207    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xr96m"]
Jul 10 23:06:29.413001 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:06:29.412962    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09245303_0104_4b38_86ed_caa866cbf92d.slice/crio-6df33a622b4c1ab487c1cdad30b0c7e5dce9a305a8a31265e502c70a2e11a7e5 WatchSource:0}: Error finding container 6df33a622b4c1ab487c1cdad30b0c7e5dce9a305a8a31265e502c70a2e11a7e5: Status 404 returned error can't find the container with id 6df33a622b4c1ab487c1cdad30b0c7e5dce9a305a8a31265e502c70a2e11a7e5
Jul 10 23:06:29.415047 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:29.415032    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 23:06:30.361732 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:30.361686    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xr96m" event={"ID":"09245303-0104-4b38-86ed-caa866cbf92d","Type":"ContainerStarted","Data":"6df33a622b4c1ab487c1cdad30b0c7e5dce9a305a8a31265e502c70a2e11a7e5"}
Jul 10 23:06:31.369159 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:31.369121    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xr96m" event={"ID":"09245303-0104-4b38-86ed-caa866cbf92d","Type":"ContainerStarted","Data":"21948735bcc321d752fe5c0caf521bae5855229c10d25bed2abbc37764b70c29"}
Jul 10 23:06:31.369619 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:31.369262    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xr96m"
Jul 10 23:06:31.386426 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:31.386308    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xr96m" podStartSLOduration=1.6641319110000001 podStartE2EDuration="3.386290184s" totalImagesPullingTime="1.722158273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:06:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:06:31.385095503 +0000 UTC m=+652.271831667" watchObservedRunningTime="2026-07-10 23:06:31.386290184 +0000 UTC m=+652.273026326"
Jul 10 23:06:34.151243 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:34.151206    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:34.206770 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:34.206730    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:34.553595 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:34.553561    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:34.603576 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:34.603542    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:38.554491 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:38.554458    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:38.664455 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:38.664422    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:39.382825 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:39.382792    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:42.376410 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:42.376347    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xr96m"
Jul 10 23:06:45.956554 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:45.956522    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:46.052530 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:46.052490    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:48.002555 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:48.002519    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:06:48.103083 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:06:48.103051    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:07:09.441584 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:09.441552    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:07:09.546492 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:09.546451    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:07:22.391500 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:22.391463    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:07:22.493136 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:22.493097    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:07:22.549687 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:22.549630    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:07:23.199669 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:23.199638    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:07:23.299987 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:23.299959    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:07:23.364379 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:23.364331    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-bkzh7"]
Jul 10 23:07:23.367797 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:23.367775    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-bkzh7"
Jul 10 23:07:23.370273 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:23.370253    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rkdp2\""
Jul 10 23:07:23.372782 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:23.372757    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bkzh7"]
Jul 10 23:07:23.490664 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:23.490629    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgjht\" (UniqueName: \"kubernetes.io/projected/b425f9df-64a2-4068-81e9-43a38ce21abf-kube-api-access-xgjht\") pod \"authorino-75d556bd77-bkzh7\" (UID: \"b425f9df-64a2-4068-81e9-43a38ce21abf\") " pod="kuadrant-system/authorino-75d556bd77-bkzh7"
Jul 10 23:07:23.591302 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:23.591265    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgjht\" (UniqueName: \"kubernetes.io/projected/b425f9df-64a2-4068-81e9-43a38ce21abf-kube-api-access-xgjht\") pod \"authorino-75d556bd77-bkzh7\" (UID: \"b425f9df-64a2-4068-81e9-43a38ce21abf\") " pod="kuadrant-system/authorino-75d556bd77-bkzh7"
Jul 10 23:07:23.599299 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:23.599272    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgjht\" (UniqueName: \"kubernetes.io/projected/b425f9df-64a2-4068-81e9-43a38ce21abf-kube-api-access-xgjht\") pod \"authorino-75d556bd77-bkzh7\" (UID: \"b425f9df-64a2-4068-81e9-43a38ce21abf\") " pod="kuadrant-system/authorino-75d556bd77-bkzh7"
Jul 10 23:07:23.679269 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:23.679226    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-bkzh7"
Jul 10 23:07:23.818882 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:23.818857    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bkzh7"]
Jul 10 23:07:23.820991 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:07:23.820965    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb425f9df_64a2_4068_81e9_43a38ce21abf.slice/crio-1a0c3208e30570ad52ddb9f1a4e676618610c32ddd27955ab2ec882039445a5d WatchSource:0}: Error finding container 1a0c3208e30570ad52ddb9f1a4e676618610c32ddd27955ab2ec882039445a5d: Status 404 returned error can't find the container with id 1a0c3208e30570ad52ddb9f1a4e676618610c32ddd27955ab2ec882039445a5d
Jul 10 23:07:24.628384 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:24.628333    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-bkzh7" event={"ID":"b425f9df-64a2-4068-81e9-43a38ce21abf","Type":"ContainerStarted","Data":"1a0c3208e30570ad52ddb9f1a4e676618610c32ddd27955ab2ec882039445a5d"}
Jul 10 23:07:27.646430 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:27.646386    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-bkzh7" event={"ID":"b425f9df-64a2-4068-81e9-43a38ce21abf","Type":"ContainerStarted","Data":"b07a15fe73fec243a5aa091a50ec6cf2d12a5114d199743d2146432db1f4f4ea"}
Jul 10 23:07:27.661375 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:27.661320    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-bkzh7" podStartSLOduration=1.5580440960000002 podStartE2EDuration="4.661302946s" totalImagesPullingTime="3.10325885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:07:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:07:27.660626929 +0000 UTC m=+708.547363074" watchObservedRunningTime="2026-07-10 23:07:27.661302946 +0000 UTC m=+708.548039092"
Jul 10 23:07:38.539007 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:38.538916    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:07:38.645589 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:38.645550    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:07:39.037585 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:39.037551    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:07:51.746148 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:51.746105    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:07:51.841588 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:51.841549    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:07:51.961380 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:51.961323    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-c5rd4"]
Jul 10 23:07:51.967510 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:51.967489    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-c5rd4"
Jul 10 23:07:51.971842 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:51.971814    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-c5rd4"]
Jul 10 23:07:52.046749 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:52.046660    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5ktp\" (UniqueName: \"kubernetes.io/projected/0aa9b921-820b-4b00-adcd-1e81175e0381-kube-api-access-l5ktp\") pod \"authorino-5d4b67c446-c5rd4\" (UID: \"0aa9b921-820b-4b00-adcd-1e81175e0381\") " pod="kuadrant-system/authorino-5d4b67c446-c5rd4"
Jul 10 23:07:52.147155 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:52.147122    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5ktp\" (UniqueName: \"kubernetes.io/projected/0aa9b921-820b-4b00-adcd-1e81175e0381-kube-api-access-l5ktp\") pod \"authorino-5d4b67c446-c5rd4\" (UID: \"0aa9b921-820b-4b00-adcd-1e81175e0381\") " pod="kuadrant-system/authorino-5d4b67c446-c5rd4"
Jul 10 23:07:52.157114 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:52.157081    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5ktp\" (UniqueName: \"kubernetes.io/projected/0aa9b921-820b-4b00-adcd-1e81175e0381-kube-api-access-l5ktp\") pod \"authorino-5d4b67c446-c5rd4\" (UID: \"0aa9b921-820b-4b00-adcd-1e81175e0381\") " pod="kuadrant-system/authorino-5d4b67c446-c5rd4"
Jul 10 23:07:52.207900 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:52.207861    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-c5rd4"]
Jul 10 23:07:52.208151 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:52.208137    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-c5rd4"
Jul 10 23:07:52.235179 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:52.235139    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7d66ccf55b-64ds5"]
Jul 10 23:07:52.238791 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:52.238768    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d66ccf55b-64ds5"
Jul 10 23:07:52.241168 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:52.241144    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 10 23:07:52.246412 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:52.246382    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d66ccf55b-64ds5"]
Jul 10 23:07:52.247849 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:52.247807    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xh74\" (UniqueName: \"kubernetes.io/projected/29e6968d-5439-47fe-a88b-0722bbb62074-kube-api-access-9xh74\") pod \"authorino-7d66ccf55b-64ds5\" (UID: \"29e6968d-5439-47fe-a88b-0722bbb62074\") " pod="kuadrant-system/authorino-7d66ccf55b-64ds5"
Jul 10 23:07:52.247993 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:52.247966    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/29e6968d-5439-47fe-a88b-0722bbb62074-tls-cert\") pod \"authorino-7d66ccf55b-64ds5\" (UID: \"29e6968d-5439-47fe-a88b-0722bbb62074\") " pod="kuadrant-system/authorino-7d66ccf55b-64ds5"
Jul 10 23:07:52.348850 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:52.348815    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/29e6968d-5439-47fe-a88b-0722bbb62074-tls-cert\") pod \"authorino-7d66ccf55b-64ds5\" (UID: \"29e6968d-5439-47fe-a88b-0722bbb62074\") " pod="kuadrant-system/authorino-7d66ccf55b-64ds5"
Jul 10 23:07:52.349039 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:52.348927    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xh74\" (UniqueName: \"kubernetes.io/projected/29e6968d-5439-47fe-a88b-0722bbb62074-kube-api-access-9xh74\") pod \"authorino-7d66ccf55b-64ds5\" (UID: \"29e6968d-5439-47fe-a88b-0722bbb62074\") " pod="kuadrant-system/authorino-7d66ccf55b-64ds5"
Jul 10 23:07:52.350454 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:52.350303    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-c5rd4"]
Jul 10 23:07:52.351636 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:52.351615    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/29e6968d-5439-47fe-a88b-0722bbb62074-tls-cert\") pod \"authorino-7d66ccf55b-64ds5\" (UID: \"29e6968d-5439-47fe-a88b-0722bbb62074\") " pod="kuadrant-system/authorino-7d66ccf55b-64ds5"
Jul 10 23:07:52.353566 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:07:52.353544    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0aa9b921_820b_4b00_adcd_1e81175e0381.slice/crio-f8f09ce4b4738a9c4ee3d79878902fb672c4ac53f4d104e95f3e9bf6de928185 WatchSource:0}: Error finding container f8f09ce4b4738a9c4ee3d79878902fb672c4ac53f4d104e95f3e9bf6de928185: Status 404 returned error can't find the container with id f8f09ce4b4738a9c4ee3d79878902fb672c4ac53f4d104e95f3e9bf6de928185
Jul 10 23:07:52.357280 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:52.357257    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xh74\" (UniqueName: \"kubernetes.io/projected/29e6968d-5439-47fe-a88b-0722bbb62074-kube-api-access-9xh74\") pod \"authorino-7d66ccf55b-64ds5\" (UID: \"29e6968d-5439-47fe-a88b-0722bbb62074\") " pod="kuadrant-system/authorino-7d66ccf55b-64ds5"
Jul 10 23:07:52.550402 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:52.550267    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d66ccf55b-64ds5"
Jul 10 23:07:52.687677 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:52.687584    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d66ccf55b-64ds5"]
Jul 10 23:07:52.689963 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:07:52.689935    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod29e6968d_5439_47fe_a88b_0722bbb62074.slice/crio-2cd1d855818577c75e4cec03fd5201e6603a6406ae4834b62fa1f377dc5cdc23 WatchSource:0}: Error finding container 2cd1d855818577c75e4cec03fd5201e6603a6406ae4834b62fa1f377dc5cdc23: Status 404 returned error can't find the container with id 2cd1d855818577c75e4cec03fd5201e6603a6406ae4834b62fa1f377dc5cdc23
Jul 10 23:07:52.766696 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:52.766658    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-c5rd4" event={"ID":"0aa9b921-820b-4b00-adcd-1e81175e0381","Type":"ContainerStarted","Data":"f8f09ce4b4738a9c4ee3d79878902fb672c4ac53f4d104e95f3e9bf6de928185"}
Jul 10 23:07:52.767774 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:52.767742    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d66ccf55b-64ds5" event={"ID":"29e6968d-5439-47fe-a88b-0722bbb62074","Type":"ContainerStarted","Data":"2cd1d855818577c75e4cec03fd5201e6603a6406ae4834b62fa1f377dc5cdc23"}
Jul 10 23:07:53.774290 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:53.774252    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d66ccf55b-64ds5" event={"ID":"29e6968d-5439-47fe-a88b-0722bbb62074","Type":"ContainerStarted","Data":"a6c7851cc7a813bb70516ab8e749b4db534b65523424117183b3d604b1e5d9e9"}
Jul 10 23:07:53.775760 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:53.775734    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-c5rd4" event={"ID":"0aa9b921-820b-4b00-adcd-1e81175e0381","Type":"ContainerStarted","Data":"e5c59c6680e3eac56c5ef836f6193da5e600df42064d423c81de815b35e1a788"}
Jul 10 23:07:53.775873 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:53.775811    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-c5rd4" podUID="0aa9b921-820b-4b00-adcd-1e81175e0381" containerName="authorino" containerID="cri-o://e5c59c6680e3eac56c5ef836f6193da5e600df42064d423c81de815b35e1a788" gracePeriod=30
Jul 10 23:07:53.790185 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:53.790129    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7d66ccf55b-64ds5" podStartSLOduration=1.50182607 podStartE2EDuration="1.790110578s" totalImagesPullingTime="288.284508ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:07:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:07:53.788434232 +0000 UTC m=+734.675170376" watchObservedRunningTime="2026-07-10 23:07:53.790110578 +0000 UTC m=+734.676846721"
Jul 10 23:07:53.803400 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:53.803324    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-c5rd4" podStartSLOduration=2.310089431 podStartE2EDuration="2.803303833s" totalImagesPullingTime="493.214402ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:07:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:07:53.801592652 +0000 UTC m=+734.688328814" watchObservedRunningTime="2026-07-10 23:07:53.803303833 +0000 UTC m=+734.690039975"
Jul 10 23:07:53.814428 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:53.814392    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bkzh7"]
Jul 10 23:07:53.814613 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:53.814580    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-bkzh7" podUID="b425f9df-64a2-4068-81e9-43a38ce21abf" containerName="authorino" containerID="cri-o://b07a15fe73fec243a5aa091a50ec6cf2d12a5114d199743d2146432db1f4f4ea" gracePeriod=30
Jul 10 23:07:54.085614 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:54.085587    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-c5rd4"
Jul 10 23:07:54.088802 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:54.088772    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-bkzh7"
Jul 10 23:07:54.166101 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:54.166054    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5ktp\" (UniqueName: \"kubernetes.io/projected/0aa9b921-820b-4b00-adcd-1e81175e0381-kube-api-access-l5ktp\") pod \"0aa9b921-820b-4b00-adcd-1e81175e0381\" (UID: \"0aa9b921-820b-4b00-adcd-1e81175e0381\") "
Jul 10 23:07:54.166311 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:54.166130    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgjht\" (UniqueName: \"kubernetes.io/projected/b425f9df-64a2-4068-81e9-43a38ce21abf-kube-api-access-xgjht\") pod \"b425f9df-64a2-4068-81e9-43a38ce21abf\" (UID: \"b425f9df-64a2-4068-81e9-43a38ce21abf\") "
Jul 10 23:07:54.168573 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:54.168540    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0aa9b921-820b-4b00-adcd-1e81175e0381-kube-api-access-l5ktp" (OuterVolumeSpecName: "kube-api-access-l5ktp") pod "0aa9b921-820b-4b00-adcd-1e81175e0381" (UID: "0aa9b921-820b-4b00-adcd-1e81175e0381"). InnerVolumeSpecName "kube-api-access-l5ktp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:07:54.168682 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:54.168605    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b425f9df-64a2-4068-81e9-43a38ce21abf-kube-api-access-xgjht" (OuterVolumeSpecName: "kube-api-access-xgjht") pod "b425f9df-64a2-4068-81e9-43a38ce21abf" (UID: "b425f9df-64a2-4068-81e9-43a38ce21abf"). InnerVolumeSpecName "kube-api-access-xgjht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:07:54.266923 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:54.266883    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgjht\" (UniqueName: \"kubernetes.io/projected/b425f9df-64a2-4068-81e9-43a38ce21abf-kube-api-access-xgjht\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:07:54.266923 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:54.266917    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5ktp\" (UniqueName: \"kubernetes.io/projected/0aa9b921-820b-4b00-adcd-1e81175e0381-kube-api-access-l5ktp\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:07:54.782171 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:54.782134    2563 generic.go:356] "Generic (PLEG): container finished" podID="b425f9df-64a2-4068-81e9-43a38ce21abf" containerID="b07a15fe73fec243a5aa091a50ec6cf2d12a5114d199743d2146432db1f4f4ea" exitCode=0
Jul 10 23:07:54.782696 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:54.782178    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-bkzh7"
Jul 10 23:07:54.782696 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:54.782241    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-bkzh7" event={"ID":"b425f9df-64a2-4068-81e9-43a38ce21abf","Type":"ContainerDied","Data":"b07a15fe73fec243a5aa091a50ec6cf2d12a5114d199743d2146432db1f4f4ea"}
Jul 10 23:07:54.782696 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:54.782287    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-bkzh7" event={"ID":"b425f9df-64a2-4068-81e9-43a38ce21abf","Type":"ContainerDied","Data":"1a0c3208e30570ad52ddb9f1a4e676618610c32ddd27955ab2ec882039445a5d"}
Jul 10 23:07:54.782696 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:54.782305    2563 scope.go:117] "RemoveContainer" containerID="b07a15fe73fec243a5aa091a50ec6cf2d12a5114d199743d2146432db1f4f4ea"
Jul 10 23:07:54.786701 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:54.786669    2563 generic.go:356] "Generic (PLEG): container finished" podID="0aa9b921-820b-4b00-adcd-1e81175e0381" containerID="e5c59c6680e3eac56c5ef836f6193da5e600df42064d423c81de815b35e1a788" exitCode=0
Jul 10 23:07:54.786862 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:54.786789    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-c5rd4" event={"ID":"0aa9b921-820b-4b00-adcd-1e81175e0381","Type":"ContainerDied","Data":"e5c59c6680e3eac56c5ef836f6193da5e600df42064d423c81de815b35e1a788"}
Jul 10 23:07:54.786862 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:54.786836    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-c5rd4"
Jul 10 23:07:54.786862 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:54.786846    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-c5rd4" event={"ID":"0aa9b921-820b-4b00-adcd-1e81175e0381","Type":"ContainerDied","Data":"f8f09ce4b4738a9c4ee3d79878902fb672c4ac53f4d104e95f3e9bf6de928185"}
Jul 10 23:07:54.795165 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:54.795144    2563 scope.go:117] "RemoveContainer" containerID="b07a15fe73fec243a5aa091a50ec6cf2d12a5114d199743d2146432db1f4f4ea"
Jul 10 23:07:54.795543 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:07:54.795522    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b07a15fe73fec243a5aa091a50ec6cf2d12a5114d199743d2146432db1f4f4ea\": container with ID starting with b07a15fe73fec243a5aa091a50ec6cf2d12a5114d199743d2146432db1f4f4ea not found: ID does not exist" containerID="b07a15fe73fec243a5aa091a50ec6cf2d12a5114d199743d2146432db1f4f4ea"
Jul 10 23:07:54.795622 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:54.795553    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b07a15fe73fec243a5aa091a50ec6cf2d12a5114d199743d2146432db1f4f4ea"} err="failed to get container status \"b07a15fe73fec243a5aa091a50ec6cf2d12a5114d199743d2146432db1f4f4ea\": rpc error: code = NotFound desc = could not find container \"b07a15fe73fec243a5aa091a50ec6cf2d12a5114d199743d2146432db1f4f4ea\": container with ID starting with b07a15fe73fec243a5aa091a50ec6cf2d12a5114d199743d2146432db1f4f4ea not found: ID does not exist"
Jul 10 23:07:54.795622 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:54.795571    2563 scope.go:117] "RemoveContainer" containerID="e5c59c6680e3eac56c5ef836f6193da5e600df42064d423c81de815b35e1a788"
Jul 10 23:07:54.805493 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:54.805463    2563 scope.go:117] "RemoveContainer" containerID="e5c59c6680e3eac56c5ef836f6193da5e600df42064d423c81de815b35e1a788"
Jul 10 23:07:54.805823 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:07:54.805803    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5c59c6680e3eac56c5ef836f6193da5e600df42064d423c81de815b35e1a788\": container with ID starting with e5c59c6680e3eac56c5ef836f6193da5e600df42064d423c81de815b35e1a788 not found: ID does not exist" containerID="e5c59c6680e3eac56c5ef836f6193da5e600df42064d423c81de815b35e1a788"
Jul 10 23:07:54.805903 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:54.805835    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5c59c6680e3eac56c5ef836f6193da5e600df42064d423c81de815b35e1a788"} err="failed to get container status \"e5c59c6680e3eac56c5ef836f6193da5e600df42064d423c81de815b35e1a788\": rpc error: code = NotFound desc = could not find container \"e5c59c6680e3eac56c5ef836f6193da5e600df42064d423c81de815b35e1a788\": container with ID starting with e5c59c6680e3eac56c5ef836f6193da5e600df42064d423c81de815b35e1a788 not found: ID does not exist"
Jul 10 23:07:54.806311 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:54.806293    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bkzh7"]
Jul 10 23:07:54.811875 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:54.811847    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bkzh7"]
Jul 10 23:07:54.822419 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:54.822385    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-c5rd4"]
Jul 10 23:07:54.824016 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:54.823992    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-c5rd4"]
Jul 10 23:07:55.556854 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:55.556821    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0aa9b921-820b-4b00-adcd-1e81175e0381" path="/var/lib/kubelet/pods/0aa9b921-820b-4b00-adcd-1e81175e0381/volumes"
Jul 10 23:07:55.557147 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:55.557133    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b425f9df-64a2-4068-81e9-43a38ce21abf" path="/var/lib/kubelet/pods/b425f9df-64a2-4068-81e9-43a38ce21abf/volumes"
Jul 10 23:07:58.149028 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:58.148994    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:07:58.247908 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:58.247877    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:07:58.321021 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:58.320987    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:07:59.012761 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:07:59.012719    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:08:01.738039 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:01.738006    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:08:01.837949 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:01.837905    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:08:34.133893 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:34.133849    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:08:34.479139 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:34.479115    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:08:34.930637 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:34.930543    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:08:35.032861 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:35.032833    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:08:35.104133 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:35.104100    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:08:45.719201 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:45.719172    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:08:45.816151 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:45.816116    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:08:51.708662 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.708625    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz"]
Jul 10 23:08:51.709087 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.709072    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b425f9df-64a2-4068-81e9-43a38ce21abf" containerName="authorino"
Jul 10 23:08:51.709131 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.709091    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b425f9df-64a2-4068-81e9-43a38ce21abf" containerName="authorino"
Jul 10 23:08:51.709131 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.709105    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aa9b921-820b-4b00-adcd-1e81175e0381" containerName="authorino"
Jul 10 23:08:51.709131 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.709113    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aa9b921-820b-4b00-adcd-1e81175e0381" containerName="authorino"
Jul 10 23:08:51.709223 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.709195    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0aa9b921-820b-4b00-adcd-1e81175e0381" containerName="authorino"
Jul 10 23:08:51.709223 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.709210    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b425f9df-64a2-4068-81e9-43a38ce21abf" containerName="authorino"
Jul 10 23:08:51.713991 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.713968    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz"
Jul 10 23:08:51.717255 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.717232    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-9b85h\""
Jul 10 23:08:51.717255 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.717250    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 10 23:08:51.717610 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.717592    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 10 23:08:51.717664 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.717593    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 10 23:08:51.721819 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.721789    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz"]
Jul 10 23:08:51.732659 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.732625    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:08:51.807558 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.807506    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/943a2f00-01e2-4006-aefb-3c1c0c6849a8-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz\" (UID: \"943a2f00-01e2-4006-aefb-3c1c0c6849a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz"
Jul 10 23:08:51.807558 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.807558    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/943a2f00-01e2-4006-aefb-3c1c0c6849a8-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz\" (UID: \"943a2f00-01e2-4006-aefb-3c1c0c6849a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz"
Jul 10 23:08:51.807798 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.807636    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/943a2f00-01e2-4006-aefb-3c1c0c6849a8-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz\" (UID: \"943a2f00-01e2-4006-aefb-3c1c0c6849a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz"
Jul 10 23:08:51.807798 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.807676    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px7jf\" (UniqueName: \"kubernetes.io/projected/943a2f00-01e2-4006-aefb-3c1c0c6849a8-kube-api-access-px7jf\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz\" (UID: \"943a2f00-01e2-4006-aefb-3c1c0c6849a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz"
Jul 10 23:08:51.807798 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.807748    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/943a2f00-01e2-4006-aefb-3c1c0c6849a8-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz\" (UID: \"943a2f00-01e2-4006-aefb-3c1c0c6849a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz"
Jul 10 23:08:51.807921 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.807795    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/943a2f00-01e2-4006-aefb-3c1c0c6849a8-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz\" (UID: \"943a2f00-01e2-4006-aefb-3c1c0c6849a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz"
Jul 10 23:08:51.909188 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.909143    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/943a2f00-01e2-4006-aefb-3c1c0c6849a8-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz\" (UID: \"943a2f00-01e2-4006-aefb-3c1c0c6849a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz"
Jul 10 23:08:51.909446 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.909201    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px7jf\" (UniqueName: \"kubernetes.io/projected/943a2f00-01e2-4006-aefb-3c1c0c6849a8-kube-api-access-px7jf\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz\" (UID: \"943a2f00-01e2-4006-aefb-3c1c0c6849a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz"
Jul 10 23:08:51.909446 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.909244    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/943a2f00-01e2-4006-aefb-3c1c0c6849a8-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz\" (UID: \"943a2f00-01e2-4006-aefb-3c1c0c6849a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz"
Jul 10 23:08:51.909446 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.909284    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/943a2f00-01e2-4006-aefb-3c1c0c6849a8-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz\" (UID: \"943a2f00-01e2-4006-aefb-3c1c0c6849a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz"
Jul 10 23:08:51.909446 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.909323    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/943a2f00-01e2-4006-aefb-3c1c0c6849a8-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz\" (UID: \"943a2f00-01e2-4006-aefb-3c1c0c6849a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz"
Jul 10 23:08:51.909446 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.909345    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/943a2f00-01e2-4006-aefb-3c1c0c6849a8-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz\" (UID: \"943a2f00-01e2-4006-aefb-3c1c0c6849a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz"
Jul 10 23:08:51.909788 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.909763    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/943a2f00-01e2-4006-aefb-3c1c0c6849a8-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz\" (UID: \"943a2f00-01e2-4006-aefb-3c1c0c6849a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz"
Jul 10 23:08:51.909861 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.909797    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/943a2f00-01e2-4006-aefb-3c1c0c6849a8-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz\" (UID: \"943a2f00-01e2-4006-aefb-3c1c0c6849a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz"
Jul 10 23:08:51.909912 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.909853    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/943a2f00-01e2-4006-aefb-3c1c0c6849a8-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz\" (UID: \"943a2f00-01e2-4006-aefb-3c1c0c6849a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz"
Jul 10 23:08:51.911676 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.911654    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/943a2f00-01e2-4006-aefb-3c1c0c6849a8-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz\" (UID: \"943a2f00-01e2-4006-aefb-3c1c0c6849a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz"
Jul 10 23:08:51.912038 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.912018    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/943a2f00-01e2-4006-aefb-3c1c0c6849a8-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz\" (UID: \"943a2f00-01e2-4006-aefb-3c1c0c6849a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz"
Jul 10 23:08:51.917453 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:51.917429    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px7jf\" (UniqueName: \"kubernetes.io/projected/943a2f00-01e2-4006-aefb-3c1c0c6849a8-kube-api-access-px7jf\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz\" (UID: \"943a2f00-01e2-4006-aefb-3c1c0c6849a8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz"
Jul 10 23:08:52.026538 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:52.026490    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz"
Jul 10 23:08:52.173458 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:52.173424    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz"]
Jul 10 23:08:52.176754 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:08:52.176722    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod943a2f00_01e2_4006_aefb_3c1c0c6849a8.slice/crio-19c5c61b1d5fd33e23eac7e4d838ad9952ca2706c093f656f08042c373381edb WatchSource:0}: Error finding container 19c5c61b1d5fd33e23eac7e4d838ad9952ca2706c093f656f08042c373381edb: Status 404 returned error can't find the container with id 19c5c61b1d5fd33e23eac7e4d838ad9952ca2706c093f656f08042c373381edb
Jul 10 23:08:53.083285 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:53.083240    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz" event={"ID":"943a2f00-01e2-4006-aefb-3c1c0c6849a8","Type":"ContainerStarted","Data":"19c5c61b1d5fd33e23eac7e4d838ad9952ca2706c093f656f08042c373381edb"}
Jul 10 23:08:54.089459 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:54.089422    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz" event={"ID":"943a2f00-01e2-4006-aefb-3c1c0c6849a8","Type":"ContainerStarted","Data":"aab82bf0971ccbe271949a72ebea5c9df61df94a127dcbc4d262921f51b3af50"}
Jul 10 23:08:54.107543 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:08:54.107482    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz" podStartSLOduration=1.91429052 podStartE2EDuration="3.107459124s" totalImagesPullingTime="1.193168604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:08:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:08:54.105432586 +0000 UTC m=+794.992168726" watchObservedRunningTime="2026-07-10 23:08:54.107459124 +0000 UTC m=+794.994195267"
Jul 10 23:09:02.027003 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:09:02.026905    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz"
Jul 10 23:09:02.027003 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:09:02.026974    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz"
Jul 10 23:09:02.039741 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:09:02.039709    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz"
Jul 10 23:09:02.138451 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:09:02.138415    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz"
Jul 10 23:09:06.146688 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:09:06.146650    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:09:27.424468 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:09:27.424436    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:09:37.823208 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:09:37.823167    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:09:43.828326 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:09:43.828287    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:09:58.024719 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:09:58.024682    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:10:39.504069 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:10:39.504043    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/ovn-acl-logging/0.log"
Jul 10 23:10:39.505063 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:10:39.505038    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/ovn-acl-logging/0.log"
Jul 10 23:11:42.479116 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:42.479078    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-684dfc7c65-gssr4"]
Jul 10 23:11:42.482610 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:42.482591    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-684dfc7c65-gssr4"
Jul 10 23:11:42.497982 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:42.497953    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-684dfc7c65-gssr4"]
Jul 10 23:11:42.576337 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:42.576297    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dftj\" (UniqueName: \"kubernetes.io/projected/78f28a98-48be-4320-ace0-1b5297b6fbe0-kube-api-access-6dftj\") pod \"authorino-684dfc7c65-gssr4\" (UID: \"78f28a98-48be-4320-ace0-1b5297b6fbe0\") " pod="kuadrant-system/authorino-684dfc7c65-gssr4"
Jul 10 23:11:42.576562 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:42.576408    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/78f28a98-48be-4320-ace0-1b5297b6fbe0-tls-cert\") pod \"authorino-684dfc7c65-gssr4\" (UID: \"78f28a98-48be-4320-ace0-1b5297b6fbe0\") " pod="kuadrant-system/authorino-684dfc7c65-gssr4"
Jul 10 23:11:42.677273 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:42.677228    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dftj\" (UniqueName: \"kubernetes.io/projected/78f28a98-48be-4320-ace0-1b5297b6fbe0-kube-api-access-6dftj\") pod \"authorino-684dfc7c65-gssr4\" (UID: \"78f28a98-48be-4320-ace0-1b5297b6fbe0\") " pod="kuadrant-system/authorino-684dfc7c65-gssr4"
Jul 10 23:11:42.677470 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:42.677311    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/78f28a98-48be-4320-ace0-1b5297b6fbe0-tls-cert\") pod \"authorino-684dfc7c65-gssr4\" (UID: \"78f28a98-48be-4320-ace0-1b5297b6fbe0\") " pod="kuadrant-system/authorino-684dfc7c65-gssr4"
Jul 10 23:11:42.679842 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:42.679812    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/78f28a98-48be-4320-ace0-1b5297b6fbe0-tls-cert\") pod \"authorino-684dfc7c65-gssr4\" (UID: \"78f28a98-48be-4320-ace0-1b5297b6fbe0\") " pod="kuadrant-system/authorino-684dfc7c65-gssr4"
Jul 10 23:11:42.685264 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:42.685241    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dftj\" (UniqueName: \"kubernetes.io/projected/78f28a98-48be-4320-ace0-1b5297b6fbe0-kube-api-access-6dftj\") pod \"authorino-684dfc7c65-gssr4\" (UID: \"78f28a98-48be-4320-ace0-1b5297b6fbe0\") " pod="kuadrant-system/authorino-684dfc7c65-gssr4"
Jul 10 23:11:42.791826 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:42.791736    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-684dfc7c65-gssr4"
Jul 10 23:11:42.934710 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:42.934665    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:11:42.935975 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:42.935952    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-684dfc7c65-gssr4"]
Jul 10 23:11:42.938404 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:11:42.938342    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod78f28a98_48be_4320_ace0_1b5297b6fbe0.slice/crio-bfddf2dc64574dbf99896fb31461bc74d3434f2ae8b3b655a968fc9f8d1c55f1 WatchSource:0}: Error finding container bfddf2dc64574dbf99896fb31461bc74d3434f2ae8b3b655a968fc9f8d1c55f1: Status 404 returned error can't find the container with id bfddf2dc64574dbf99896fb31461bc74d3434f2ae8b3b655a968fc9f8d1c55f1
Jul 10 23:11:42.939733 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:42.939714    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 23:11:43.929246 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:43.929213    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-684dfc7c65-gssr4" event={"ID":"78f28a98-48be-4320-ace0-1b5297b6fbe0","Type":"ContainerStarted","Data":"c94b42c6ffa37ec2d49ded84fdfa300c4851b132b9293824d8213e17d5f7ee55"}
Jul 10 23:11:43.929671 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:43.929255    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-684dfc7c65-gssr4" event={"ID":"78f28a98-48be-4320-ace0-1b5297b6fbe0","Type":"ContainerStarted","Data":"bfddf2dc64574dbf99896fb31461bc74d3434f2ae8b3b655a968fc9f8d1c55f1"}
Jul 10 23:11:43.947128 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:43.947072    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-684dfc7c65-gssr4" podStartSLOduration=1.4135369629999999 podStartE2EDuration="1.947056232s" totalImagesPullingTime="533.519269ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:11:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:11:43.946684261 +0000 UTC m=+964.833420403" watchObservedRunningTime="2026-07-10 23:11:43.947056232 +0000 UTC m=+964.833792375"
Jul 10 23:11:43.977582 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:43.977545    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:11:43.979390 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:43.979348    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7d66ccf55b-64ds5"]
Jul 10 23:11:43.979583 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:43.979549    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7d66ccf55b-64ds5" podUID="29e6968d-5439-47fe-a88b-0722bbb62074" containerName="authorino" containerID="cri-o://a6c7851cc7a813bb70516ab8e749b4db534b65523424117183b3d604b1e5d9e9" gracePeriod=30
Jul 10 23:11:44.134356 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:44.134320    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:11:44.232236 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:44.232203    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:11:44.246105 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:44.246067    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d66ccf55b-64ds5"
Jul 10 23:11:44.392966 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:44.392927    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xh74\" (UniqueName: \"kubernetes.io/projected/29e6968d-5439-47fe-a88b-0722bbb62074-kube-api-access-9xh74\") pod \"29e6968d-5439-47fe-a88b-0722bbb62074\" (UID: \"29e6968d-5439-47fe-a88b-0722bbb62074\") "
Jul 10 23:11:44.393217 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:44.392986    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/29e6968d-5439-47fe-a88b-0722bbb62074-tls-cert\") pod \"29e6968d-5439-47fe-a88b-0722bbb62074\" (UID: \"29e6968d-5439-47fe-a88b-0722bbb62074\") "
Jul 10 23:11:44.395795 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:44.395753    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29e6968d-5439-47fe-a88b-0722bbb62074-kube-api-access-9xh74" (OuterVolumeSpecName: "kube-api-access-9xh74") pod "29e6968d-5439-47fe-a88b-0722bbb62074" (UID: "29e6968d-5439-47fe-a88b-0722bbb62074"). InnerVolumeSpecName "kube-api-access-9xh74". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:11:44.405215 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:44.405179    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29e6968d-5439-47fe-a88b-0722bbb62074-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "29e6968d-5439-47fe-a88b-0722bbb62074" (UID: "29e6968d-5439-47fe-a88b-0722bbb62074"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:11:44.494858 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:44.494816    2563 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/29e6968d-5439-47fe-a88b-0722bbb62074-tls-cert\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:11:44.494858 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:44.494857    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xh74\" (UniqueName: \"kubernetes.io/projected/29e6968d-5439-47fe-a88b-0722bbb62074-kube-api-access-9xh74\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:11:44.936944 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:44.936849    2563 generic.go:356] "Generic (PLEG): container finished" podID="29e6968d-5439-47fe-a88b-0722bbb62074" containerID="a6c7851cc7a813bb70516ab8e749b4db534b65523424117183b3d604b1e5d9e9" exitCode=0
Jul 10 23:11:44.936944 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:44.936894    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d66ccf55b-64ds5"
Jul 10 23:11:44.937497 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:44.936935    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d66ccf55b-64ds5" event={"ID":"29e6968d-5439-47fe-a88b-0722bbb62074","Type":"ContainerDied","Data":"a6c7851cc7a813bb70516ab8e749b4db534b65523424117183b3d604b1e5d9e9"}
Jul 10 23:11:44.937497 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:44.936990    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d66ccf55b-64ds5" event={"ID":"29e6968d-5439-47fe-a88b-0722bbb62074","Type":"ContainerDied","Data":"2cd1d855818577c75e4cec03fd5201e6603a6406ae4834b62fa1f377dc5cdc23"}
Jul 10 23:11:44.937497 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:44.937015    2563 scope.go:117] "RemoveContainer" containerID="a6c7851cc7a813bb70516ab8e749b4db534b65523424117183b3d604b1e5d9e9"
Jul 10 23:11:44.947724 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:44.947702    2563 scope.go:117] "RemoveContainer" containerID="a6c7851cc7a813bb70516ab8e749b4db534b65523424117183b3d604b1e5d9e9"
Jul 10 23:11:44.948102 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:11:44.948077    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6c7851cc7a813bb70516ab8e749b4db534b65523424117183b3d604b1e5d9e9\": container with ID starting with a6c7851cc7a813bb70516ab8e749b4db534b65523424117183b3d604b1e5d9e9 not found: ID does not exist" containerID="a6c7851cc7a813bb70516ab8e749b4db534b65523424117183b3d604b1e5d9e9"
Jul 10 23:11:44.948182 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:44.948118    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6c7851cc7a813bb70516ab8e749b4db534b65523424117183b3d604b1e5d9e9"} err="failed to get container status \"a6c7851cc7a813bb70516ab8e749b4db534b65523424117183b3d604b1e5d9e9\": rpc error: code = NotFound desc = could not find container \"a6c7851cc7a813bb70516ab8e749b4db534b65523424117183b3d604b1e5d9e9\": container with ID starting with a6c7851cc7a813bb70516ab8e749b4db534b65523424117183b3d604b1e5d9e9 not found: ID does not exist"
Jul 10 23:11:44.959826 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:44.959793    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7d66ccf55b-64ds5"]
Jul 10 23:11:44.963867 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:44.963839    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7d66ccf55b-64ds5"]
Jul 10 23:11:45.558507 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:11:45.558473    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29e6968d-5439-47fe-a88b-0722bbb62074" path="/var/lib/kubelet/pods/29e6968d-5439-47fe-a88b-0722bbb62074/volumes"
Jul 10 23:12:36.020768 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:12:36.020731    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:12:55.350726 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:12:55.350688    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:12:56.785384 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:12:56.785330    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:13:07.569603 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:13:07.569574    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:13:07.631118 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:13:07.631087    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:13:10.737778 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:13:10.737740    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:13:26.372070 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:13:26.372036    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:13:26.473308 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:13:26.473274    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:13:30.401397 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:13:30.401274    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:13:30.503013 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:13:30.502981    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:13:30.638422 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:13:30.638386    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:13:36.282486 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:13:36.282440    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:13:36.383741 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:13:36.383710    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:13:36.484443 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:13:36.484412    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:13:36.635740 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:13:36.635656    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:13:42.390708 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:13:42.390667    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:13:42.455129 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:13:42.455090    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:13:57.323015 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:13:57.322979    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:15:00.143908 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:15:00.143816    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728755-gxr5b"]
Jul 10 23:15:00.144411 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:15:00.144251    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29e6968d-5439-47fe-a88b-0722bbb62074" containerName="authorino"
Jul 10 23:15:00.144411 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:15:00.144266    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="29e6968d-5439-47fe-a88b-0722bbb62074" containerName="authorino"
Jul 10 23:15:00.144411 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:15:00.144329    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="29e6968d-5439-47fe-a88b-0722bbb62074" containerName="authorino"
Jul 10 23:15:00.147160 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:15:00.147139    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728755-gxr5b"
Jul 10 23:15:00.149651 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:15:00.149624    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p89rl\""
Jul 10 23:15:00.157481 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:15:00.157456    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728755-gxr5b"
Jul 10 23:15:00.161680 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:15:00.161650    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728755-gxr5b"]
Jul 10 23:15:00.296972 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:15:00.296785    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728755-gxr5b"]
Jul 10 23:15:00.300131 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:15:00.300103    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4012ccae_c11e_4b22_8fb0_9dbbba4b5e21.slice/crio-305239bc2aec6731357998459a6db32f0e8947879a8c53746e8afded7624749a WatchSource:0}: Error finding container 305239bc2aec6731357998459a6db32f0e8947879a8c53746e8afded7624749a: Status 404 returned error can't find the container with id 305239bc2aec6731357998459a6db32f0e8947879a8c53746e8afded7624749a
Jul 10 23:15:00.903321 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:15:00.903274    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728755-gxr5b" event={"ID":"4012ccae-c11e-4b22-8fb0-9dbbba4b5e21","Type":"ContainerStarted","Data":"305239bc2aec6731357998459a6db32f0e8947879a8c53746e8afded7624749a"}
Jul 10 23:15:03.921637 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:15:03.921598    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728755-gxr5b" event={"ID":"4012ccae-c11e-4b22-8fb0-9dbbba4b5e21","Type":"ContainerStarted","Data":"923863f5ac386a25baba870fa25b23b0eb5305b8fd90293a4b38d9d63d358589"}
Jul 10 23:15:03.935547 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:15:03.935474    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728755-gxr5b" podStartSLOduration=1.258071623 podStartE2EDuration="3.935458559s" totalImagesPullingTime="2.677386936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:15:03.934552254 +0000 UTC m=+1164.821288414" watchObservedRunningTime="2026-07-10 23:15:03.935458559 +0000 UTC m=+1164.822194702"
Jul 10 23:15:39.543499 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:15:39.543465    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/ovn-acl-logging/0.log"
Jul 10 23:15:39.544923 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:15:39.544898    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/ovn-acl-logging/0.log"
Jul 10 23:15:42.917310 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:15:42.917275    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:16:00.010472 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:16:00.010432    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728755-gxr5b"]
Jul 10 23:16:00.010909 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:16:00.010644    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728755-gxr5b" podUID="4012ccae-c11e-4b22-8fb0-9dbbba4b5e21" containerName="cleanup" containerID="cri-o://923863f5ac386a25baba870fa25b23b0eb5305b8fd90293a4b38d9d63d358589" gracePeriod=30
Jul 10 23:16:30.200092 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:16:30.200063    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728755-gxr5b"
Jul 10 23:16:30.360768 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:16:30.360677    2563 generic.go:356] "Generic (PLEG): container finished" podID="4012ccae-c11e-4b22-8fb0-9dbbba4b5e21" containerID="923863f5ac386a25baba870fa25b23b0eb5305b8fd90293a4b38d9d63d358589" exitCode=137
Jul 10 23:16:30.360768 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:16:30.360718    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728755-gxr5b"
Jul 10 23:16:30.360768 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:16:30.360742    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728755-gxr5b" event={"ID":"4012ccae-c11e-4b22-8fb0-9dbbba4b5e21","Type":"ContainerDied","Data":"923863f5ac386a25baba870fa25b23b0eb5305b8fd90293a4b38d9d63d358589"}
Jul 10 23:16:30.360994 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:16:30.360792    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728755-gxr5b" event={"ID":"4012ccae-c11e-4b22-8fb0-9dbbba4b5e21","Type":"ContainerDied","Data":"305239bc2aec6731357998459a6db32f0e8947879a8c53746e8afded7624749a"}
Jul 10 23:16:30.360994 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:16:30.360817    2563 scope.go:117] "RemoveContainer" containerID="923863f5ac386a25baba870fa25b23b0eb5305b8fd90293a4b38d9d63d358589"
Jul 10 23:16:30.370177 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:16:30.370157    2563 scope.go:117] "RemoveContainer" containerID="923863f5ac386a25baba870fa25b23b0eb5305b8fd90293a4b38d9d63d358589"
Jul 10 23:16:30.370502 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:16:30.370482    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"923863f5ac386a25baba870fa25b23b0eb5305b8fd90293a4b38d9d63d358589\": container with ID starting with 923863f5ac386a25baba870fa25b23b0eb5305b8fd90293a4b38d9d63d358589 not found: ID does not exist" containerID="923863f5ac386a25baba870fa25b23b0eb5305b8fd90293a4b38d9d63d358589"
Jul 10 23:16:30.370552 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:16:30.370515    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"923863f5ac386a25baba870fa25b23b0eb5305b8fd90293a4b38d9d63d358589"} err="failed to get container status \"923863f5ac386a25baba870fa25b23b0eb5305b8fd90293a4b38d9d63d358589\": rpc error: code = NotFound desc = could not find container \"923863f5ac386a25baba870fa25b23b0eb5305b8fd90293a4b38d9d63d358589\": container with ID starting with 923863f5ac386a25baba870fa25b23b0eb5305b8fd90293a4b38d9d63d358589 not found: ID does not exist"
Jul 10 23:16:30.381589 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:16:30.381559    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728755-gxr5b"]
Jul 10 23:16:30.385455 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:16:30.385426    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728755-gxr5b"]
Jul 10 23:16:31.556528 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:16:31.556486    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4012ccae-c11e-4b22-8fb0-9dbbba4b5e21" path="/var/lib/kubelet/pods/4012ccae-c11e-4b22-8fb0-9dbbba4b5e21/volumes"
Jul 10 23:18:07.593451 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:18:07.593341    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:19:04.794433 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:19:04.794389    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:19:59.726256 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:19:59.726175    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:20:19.315662 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:20:19.315626    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:20:39.583907 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:20:39.583878    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/ovn-acl-logging/0.log"
Jul 10 23:20:39.584509 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:20:39.584242    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/ovn-acl-logging/0.log"
Jul 10 23:24:23.957191 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:23.957153    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:24:31.509193 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:31.509156    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:24:39.781752 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:39.781717    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:24:44.851170 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:44.851129    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:24:47.744516 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:47.744482    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:24:47.841346 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:47.841314    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:24:48.182617 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:48.182504    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:24:53.848601 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:53.848557    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:24:53.981107 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:53.981074    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:24:54.094643 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:54.094613    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:24:54.194914 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:54.194816    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:24:55.238420 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:55.238378    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:24:55.318221 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:55.318189    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7"]
Jul 10 23:24:55.318221 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:55.318213    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:24:55.318630 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:55.318613    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4012ccae-c11e-4b22-8fb0-9dbbba4b5e21" containerName="cleanup"
Jul 10 23:24:55.318630 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:55.318630    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4012ccae-c11e-4b22-8fb0-9dbbba4b5e21" containerName="cleanup"
Jul 10 23:24:55.318750 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:55.318697    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4012ccae-c11e-4b22-8fb0-9dbbba4b5e21" containerName="cleanup"
Jul 10 23:24:55.321750 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:55.321728    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7"
Jul 10 23:24:55.324648 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:55.324620    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fxg5k\""
Jul 10 23:24:55.324648 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:55.324637    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-6ec12ec4-serving-cert\""
Jul 10 23:24:55.324946 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:55.324930    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 23:24:55.325569 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:55.325549    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 10 23:24:55.337804 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:55.337777    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7"]
Jul 10 23:24:55.385631 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:55.385598    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:24:55.422019 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:55.421980    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e2ff793-5550-4067-bda4-233a7586c3a6-maas-api-tls\") pod \"maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7\" (UID: \"5e2ff793-5550-4067-bda4-233a7586c3a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7"
Jul 10 23:24:55.422199 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:55.422126    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8jg7\" (UniqueName: \"kubernetes.io/projected/5e2ff793-5550-4067-bda4-233a7586c3a6-kube-api-access-b8jg7\") pod \"maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7\" (UID: \"5e2ff793-5550-4067-bda4-233a7586c3a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7"
Jul 10 23:24:55.422199 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:55.422155    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e2ff793-5550-4067-bda4-233a7586c3a6-openshift-service-ca\") pod \"maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7\" (UID: \"5e2ff793-5550-4067-bda4-233a7586c3a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7"
Jul 10 23:24:55.523345 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:55.523237    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8jg7\" (UniqueName: \"kubernetes.io/projected/5e2ff793-5550-4067-bda4-233a7586c3a6-kube-api-access-b8jg7\") pod \"maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7\" (UID: \"5e2ff793-5550-4067-bda4-233a7586c3a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7"
Jul 10 23:24:55.523345 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:55.523287    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e2ff793-5550-4067-bda4-233a7586c3a6-openshift-service-ca\") pod \"maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7\" (UID: \"5e2ff793-5550-4067-bda4-233a7586c3a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7"
Jul 10 23:24:55.523345 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:55.523322    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e2ff793-5550-4067-bda4-233a7586c3a6-maas-api-tls\") pod \"maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7\" (UID: \"5e2ff793-5550-4067-bda4-233a7586c3a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7"
Jul 10 23:24:55.523713 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:24:55.523472    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-6ec12ec4-serving-cert: secret "maas-api-e2e-ait-6ec12ec4-serving-cert" not found
Jul 10 23:24:55.523713 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:24:55.523542    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e2ff793-5550-4067-bda4-233a7586c3a6-maas-api-tls podName:5e2ff793-5550-4067-bda4-233a7586c3a6 nodeName:}" failed. No retries permitted until 2026-07-10 23:24:56.02352167 +0000 UTC m=+1756.910257791 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5e2ff793-5550-4067-bda4-233a7586c3a6-maas-api-tls") pod "maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7" (UID: "5e2ff793-5550-4067-bda4-233a7586c3a6") : secret "maas-api-e2e-ait-6ec12ec4-serving-cert" not found
Jul 10 23:24:55.523974 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:55.523951    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e2ff793-5550-4067-bda4-233a7586c3a6-openshift-service-ca\") pod \"maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7\" (UID: \"5e2ff793-5550-4067-bda4-233a7586c3a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7"
Jul 10 23:24:55.531977 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:55.531941    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8jg7\" (UniqueName: \"kubernetes.io/projected/5e2ff793-5550-4067-bda4-233a7586c3a6-kube-api-access-b8jg7\") pod \"maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7\" (UID: \"5e2ff793-5550-4067-bda4-233a7586c3a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7"
Jul 10 23:24:55.650725 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:55.650692    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:24:55.891026 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:55.890943    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:24:55.991957 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:55.991912    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:24:56.026058 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:56.026030    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e2ff793-5550-4067-bda4-233a7586c3a6-maas-api-tls\") pod \"maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7\" (UID: \"5e2ff793-5550-4067-bda4-233a7586c3a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7"
Jul 10 23:24:56.028755 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:56.028722    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e2ff793-5550-4067-bda4-233a7586c3a6-maas-api-tls\") pod \"maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7\" (UID: \"5e2ff793-5550-4067-bda4-233a7586c3a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7"
Jul 10 23:24:56.233165 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:56.233132    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7"
Jul 10 23:24:56.375243 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:56.375141    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7"]
Jul 10 23:24:56.377698 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:24:56.377673    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e2ff793_5550_4067_bda4_233a7586c3a6.slice/crio-98be805c15d50c0d0dd4824b552946e0a5d69902def766e09f2708e205f68dcb WatchSource:0}: Error finding container 98be805c15d50c0d0dd4824b552946e0a5d69902def766e09f2708e205f68dcb: Status 404 returned error can't find the container with id 98be805c15d50c0d0dd4824b552946e0a5d69902def766e09f2708e205f68dcb
Jul 10 23:24:56.379215 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:56.379188    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 23:24:56.858150 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:56.858108    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7" event={"ID":"5e2ff793-5550-4067-bda4-233a7586c3a6","Type":"ContainerStarted","Data":"98be805c15d50c0d0dd4824b552946e0a5d69902def766e09f2708e205f68dcb"}
Jul 10 23:24:57.816025 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:57.815501    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:24:58.871585 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:58.871550    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7" event={"ID":"5e2ff793-5550-4067-bda4-233a7586c3a6","Type":"ContainerStarted","Data":"f587272105b540021c0b718fb67af1be489b159d3d1e7eb31fdfa71e7edf6a46"}
Jul 10 23:24:58.872099 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:58.871676    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7"
Jul 10 23:24:58.888531 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:24:58.888475    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7" podStartSLOduration=2.186459911 podStartE2EDuration="3.888457105s" totalImagesPullingTime="1.701997194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:24:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:24:58.886605991 +0000 UTC m=+1759.773342135" watchObservedRunningTime="2026-07-10 23:24:58.888457105 +0000 UTC m=+1759.775193247"
Jul 10 23:25:00.283274 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:00.283238    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:00.470809 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:00.470777    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:00.569541 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:00.569451    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:00.684949 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:00.684912    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7"]
Jul 10 23:25:00.752279 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:00.752247    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:00.881871 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:00.881764    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7" podUID="5e2ff793-5550-4067-bda4-233a7586c3a6" containerName="maas-api" containerID="cri-o://f587272105b540021c0b718fb67af1be489b159d3d1e7eb31fdfa71e7edf6a46" gracePeriod=30
Jul 10 23:25:01.137219 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:01.137133    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7"
Jul 10 23:25:01.169535 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:01.169504    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e2ff793-5550-4067-bda4-233a7586c3a6-openshift-service-ca\") pod \"5e2ff793-5550-4067-bda4-233a7586c3a6\" (UID: \"5e2ff793-5550-4067-bda4-233a7586c3a6\") "
Jul 10 23:25:01.169704 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:01.169544    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e2ff793-5550-4067-bda4-233a7586c3a6-maas-api-tls\") pod \"5e2ff793-5550-4067-bda4-233a7586c3a6\" (UID: \"5e2ff793-5550-4067-bda4-233a7586c3a6\") "
Jul 10 23:25:01.169704 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:01.169582    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8jg7\" (UniqueName: \"kubernetes.io/projected/5e2ff793-5550-4067-bda4-233a7586c3a6-kube-api-access-b8jg7\") pod \"5e2ff793-5550-4067-bda4-233a7586c3a6\" (UID: \"5e2ff793-5550-4067-bda4-233a7586c3a6\") "
Jul 10 23:25:01.169912 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:01.169889    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e2ff793-5550-4067-bda4-233a7586c3a6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5e2ff793-5550-4067-bda4-233a7586c3a6" (UID: "5e2ff793-5550-4067-bda4-233a7586c3a6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:25:01.171967 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:01.171930    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e2ff793-5550-4067-bda4-233a7586c3a6-kube-api-access-b8jg7" (OuterVolumeSpecName: "kube-api-access-b8jg7") pod "5e2ff793-5550-4067-bda4-233a7586c3a6" (UID: "5e2ff793-5550-4067-bda4-233a7586c3a6"). InnerVolumeSpecName "kube-api-access-b8jg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:25:01.171967 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:01.171937    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e2ff793-5550-4067-bda4-233a7586c3a6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5e2ff793-5550-4067-bda4-233a7586c3a6" (UID: "5e2ff793-5550-4067-bda4-233a7586c3a6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:25:01.270432 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:01.270352    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e2ff793-5550-4067-bda4-233a7586c3a6-openshift-service-ca\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:25:01.270432 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:01.270418    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e2ff793-5550-4067-bda4-233a7586c3a6-maas-api-tls\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:25:01.270432 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:01.270432    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8jg7\" (UniqueName: \"kubernetes.io/projected/5e2ff793-5550-4067-bda4-233a7586c3a6-kube-api-access-b8jg7\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:25:01.887848 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:01.887755    2563 generic.go:356] "Generic (PLEG): container finished" podID="5e2ff793-5550-4067-bda4-233a7586c3a6" containerID="f587272105b540021c0b718fb67af1be489b159d3d1e7eb31fdfa71e7edf6a46" exitCode=0
Jul 10 23:25:01.887848 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:01.887809    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7"
Jul 10 23:25:01.887848 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:01.887830    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7" event={"ID":"5e2ff793-5550-4067-bda4-233a7586c3a6","Type":"ContainerDied","Data":"f587272105b540021c0b718fb67af1be489b159d3d1e7eb31fdfa71e7edf6a46"}
Jul 10 23:25:01.888345 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:01.887884    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7" event={"ID":"5e2ff793-5550-4067-bda4-233a7586c3a6","Type":"ContainerDied","Data":"98be805c15d50c0d0dd4824b552946e0a5d69902def766e09f2708e205f68dcb"}
Jul 10 23:25:01.888345 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:01.887923    2563 scope.go:117] "RemoveContainer" containerID="f587272105b540021c0b718fb67af1be489b159d3d1e7eb31fdfa71e7edf6a46"
Jul 10 23:25:01.896976 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:01.896954    2563 scope.go:117] "RemoveContainer" containerID="f587272105b540021c0b718fb67af1be489b159d3d1e7eb31fdfa71e7edf6a46"
Jul 10 23:25:01.897303 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:25:01.897280    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f587272105b540021c0b718fb67af1be489b159d3d1e7eb31fdfa71e7edf6a46\": container with ID starting with f587272105b540021c0b718fb67af1be489b159d3d1e7eb31fdfa71e7edf6a46 not found: ID does not exist" containerID="f587272105b540021c0b718fb67af1be489b159d3d1e7eb31fdfa71e7edf6a46"
Jul 10 23:25:01.897349 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:01.897314    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f587272105b540021c0b718fb67af1be489b159d3d1e7eb31fdfa71e7edf6a46"} err="failed to get container status \"f587272105b540021c0b718fb67af1be489b159d3d1e7eb31fdfa71e7edf6a46\": rpc error: code = NotFound desc = could not find container \"f587272105b540021c0b718fb67af1be489b159d3d1e7eb31fdfa71e7edf6a46\": container with ID starting with f587272105b540021c0b718fb67af1be489b159d3d1e7eb31fdfa71e7edf6a46 not found: ID does not exist"
Jul 10 23:25:01.903597 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:01.903561    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7"]
Jul 10 23:25:01.905643 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:01.905614    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6ec12ec4-5bf67fcd76-xgvp7"]
Jul 10 23:25:03.556600 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:03.556567    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e2ff793-5550-4067-bda4-233a7586c3a6" path="/var/lib/kubelet/pods/5e2ff793-5550-4067-bda4-233a7586c3a6/volumes"
Jul 10 23:25:03.621541 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:03.621516    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:06.261729 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.261693    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:06.323530 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.323502    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:06.432946 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.432915    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"]
Jul 10 23:25:06.433285 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.433272    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e2ff793-5550-4067-bda4-233a7586c3a6" containerName="maas-api"
Jul 10 23:25:06.433332 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.433286    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e2ff793-5550-4067-bda4-233a7586c3a6" containerName="maas-api"
Jul 10 23:25:06.433427 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.433416    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e2ff793-5550-4067-bda4-233a7586c3a6" containerName="maas-api"
Jul 10 23:25:06.437701 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.437680    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.440469 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.440446    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-76896fa5-openshift-default-dockercfg-zkdvn\""
Jul 10 23:25:06.440469 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.440464    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 10 23:25:06.448044 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.447880    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"]
Jul 10 23:25:06.466950 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.466892    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:06.517426 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.517278    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19dade5d-4a01-4adb-af7b-12bb211ef919-istiod-ca-cert\") pod \"e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") " pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.517426 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.517353    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-credential-socket\") pod \"e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") " pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.517670 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.517448    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-workload-certs\") pod \"e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") " pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.517670 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.517486    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-istio-envoy\") pod \"e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") " pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.517670 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.517514    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19dade5d-4a01-4adb-af7b-12bb211ef919-istio-podinfo\") pod \"e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") " pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.517670 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.517608    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djh9t\" (UniqueName: \"kubernetes.io/projected/19dade5d-4a01-4adb-af7b-12bb211ef919-kube-api-access-djh9t\") pod \"e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") " pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.517869 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.517681    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-istio-data\") pod \"e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") " pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.517869 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.517735    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19dade5d-4a01-4adb-af7b-12bb211ef919-istio-token\") pod \"e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") " pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.517869 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.517770    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-workload-socket\") pod \"e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") " pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.518090 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.518067    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:06.619161 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.619117    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-istio-data\") pod \"e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") " pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.619329 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.619174    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19dade5d-4a01-4adb-af7b-12bb211ef919-istio-token\") pod \"e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") " pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.619329 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.619221    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-workload-socket\") pod \"e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") " pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.619329 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.619255    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19dade5d-4a01-4adb-af7b-12bb211ef919-istiod-ca-cert\") pod \"e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") " pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.619329 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.619293    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-credential-socket\") pod \"e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") " pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.619561 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.619450    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-workload-certs\") pod \"e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") " pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.619561 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.619509    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-istio-envoy\") pod \"e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") " pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.619561 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.619540    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19dade5d-4a01-4adb-af7b-12bb211ef919-istio-podinfo\") pod \"e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") " pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.619756 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.619597    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djh9t\" (UniqueName: \"kubernetes.io/projected/19dade5d-4a01-4adb-af7b-12bb211ef919-kube-api-access-djh9t\") pod \"e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") " pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.619814 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.619770    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-workload-socket\") pod \"e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") " pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.619814 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.619797    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-workload-certs\") pod \"e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") " pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.619927 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.619853    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-credential-socket\") pod \"e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") " pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.619987 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.619960    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-istio-data\") pod \"e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") " pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.620195 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.620173    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19dade5d-4a01-4adb-af7b-12bb211ef919-istiod-ca-cert\") pod \"e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") " pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.621799 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.621781    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-istio-envoy\") pod \"e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") " pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.622039 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.622020    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19dade5d-4a01-4adb-af7b-12bb211ef919-istio-podinfo\") pod \"e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") " pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.627379 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.627339    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19dade5d-4a01-4adb-af7b-12bb211ef919-istio-token\") pod \"e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") " pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.627515 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.627449    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djh9t\" (UniqueName: \"kubernetes.io/projected/19dade5d-4a01-4adb-af7b-12bb211ef919-kube-api-access-djh9t\") pod \"e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") " pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.750582 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.750534    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:06.898575 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.898547    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"]
Jul 10 23:25:06.900938 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:25:06.900911    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19dade5d_4a01_4adb_af7b_12bb211ef919.slice/crio-3bcac1f508e2023be6b32e6eb2097830e85b921f143bf8d475ac22d57d6537b6 WatchSource:0}: Error finding container 3bcac1f508e2023be6b32e6eb2097830e85b921f143bf8d475ac22d57d6537b6: Status 404 returned error can't find the container with id 3bcac1f508e2023be6b32e6eb2097830e85b921f143bf8d475ac22d57d6537b6
Jul 10 23:25:06.915637 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:06.915600    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb" event={"ID":"19dade5d-4a01-4adb-af7b-12bb211ef919","Type":"ContainerStarted","Data":"3bcac1f508e2023be6b32e6eb2097830e85b921f143bf8d475ac22d57d6537b6"}
Jul 10 23:25:07.626989 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:07.626944    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:07.701461 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:07.701387    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:07.766433 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:07.766324    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:08.205086 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:08.205023    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:08.306290 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:08.306256    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:08.370988 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:08.370888    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:09.553203 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:09.553155    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:25:09.553560 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:09.553234    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:25:09.553560 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:09.553281    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:25:09.754499 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:09.754466    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:09.934479 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:09.934389    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb" event={"ID":"19dade5d-4a01-4adb-af7b-12bb211ef919","Type":"ContainerStarted","Data":"5d1e4867a89b5b1a5fc9cba6d1d398586dfd2dbf4c732338708f12d9886b6c5b"}
Jul 10 23:25:09.955679 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:09.955631    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb" podStartSLOduration=1.305585632 podStartE2EDuration="3.955614012s" totalImagesPullingTime="2.65002838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:25:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:25:09.952264578 +0000 UTC m=+1770.839000723" watchObservedRunningTime="2026-07-10 23:25:09.955614012 +0000 UTC m=+1770.842350154"
Jul 10 23:25:10.751659 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:10.751621    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:10.756790 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:10.756763    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:10.940883 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:10.940851    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:10.941866 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:10.941840    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:12.757690 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:12.757648    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:13.185018 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:13.184930    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:18.823568 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:18.823523    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"]
Jul 10 23:25:18.824022 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:18.823750    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb" podUID="19dade5d-4a01-4adb-af7b-12bb211ef919" containerName="istio-proxy" containerID="cri-o://5d1e4867a89b5b1a5fc9cba6d1d398586dfd2dbf4c732338708f12d9886b6c5b" gracePeriod=30
Jul 10 23:25:18.845698 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:18.845661    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:18.957664 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:18.957623    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:19.186187 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.186153    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:19.249865 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.249835    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-istio-envoy\") pod \"19dade5d-4a01-4adb-af7b-12bb211ef919\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") "
Jul 10 23:25:19.250049 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.249875    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-credential-socket\") pod \"19dade5d-4a01-4adb-af7b-12bb211ef919\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") "
Jul 10 23:25:19.250049 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.249902    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-workload-certs\") pod \"19dade5d-4a01-4adb-af7b-12bb211ef919\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") "
Jul 10 23:25:19.250049 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.249958    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19dade5d-4a01-4adb-af7b-12bb211ef919-istiod-ca-cert\") pod \"19dade5d-4a01-4adb-af7b-12bb211ef919\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") "
Jul 10 23:25:19.250049 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.250011    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-workload-socket\") pod \"19dade5d-4a01-4adb-af7b-12bb211ef919\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") "
Jul 10 23:25:19.250049 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.250037    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-istio-data\") pod \"19dade5d-4a01-4adb-af7b-12bb211ef919\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") "
Jul 10 23:25:19.250304 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.250106    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19dade5d-4a01-4adb-af7b-12bb211ef919-istio-podinfo\") pod \"19dade5d-4a01-4adb-af7b-12bb211ef919\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") "
Jul 10 23:25:19.250304 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.250141    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19dade5d-4a01-4adb-af7b-12bb211ef919-istio-token\") pod \"19dade5d-4a01-4adb-af7b-12bb211ef919\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") "
Jul 10 23:25:19.250304 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.250179    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djh9t\" (UniqueName: \"kubernetes.io/projected/19dade5d-4a01-4adb-af7b-12bb211ef919-kube-api-access-djh9t\") pod \"19dade5d-4a01-4adb-af7b-12bb211ef919\" (UID: \"19dade5d-4a01-4adb-af7b-12bb211ef919\") "
Jul 10 23:25:19.250304 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.250195    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "19dade5d-4a01-4adb-af7b-12bb211ef919" (UID: "19dade5d-4a01-4adb-af7b-12bb211ef919"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:25:19.250304 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.250210    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "19dade5d-4a01-4adb-af7b-12bb211ef919" (UID: "19dade5d-4a01-4adb-af7b-12bb211ef919"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:25:19.250578 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.250473    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-credential-socket\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:25:19.250578 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.250490    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-workload-certs\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:25:19.251211 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.251179    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-istio-data" (OuterVolumeSpecName: "istio-data") pod "19dade5d-4a01-4adb-af7b-12bb211ef919" (UID: "19dade5d-4a01-4adb-af7b-12bb211ef919"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:25:19.251615 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.251590    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19dade5d-4a01-4adb-af7b-12bb211ef919-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "19dade5d-4a01-4adb-af7b-12bb211ef919" (UID: "19dade5d-4a01-4adb-af7b-12bb211ef919"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:25:19.251725 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.251676    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "19dade5d-4a01-4adb-af7b-12bb211ef919" (UID: "19dade5d-4a01-4adb-af7b-12bb211ef919"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:25:19.252711 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.252688    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19dade5d-4a01-4adb-af7b-12bb211ef919-kube-api-access-djh9t" (OuterVolumeSpecName: "kube-api-access-djh9t") pod "19dade5d-4a01-4adb-af7b-12bb211ef919" (UID: "19dade5d-4a01-4adb-af7b-12bb211ef919"). InnerVolumeSpecName "kube-api-access-djh9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:25:19.252807 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.252794    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "19dade5d-4a01-4adb-af7b-12bb211ef919" (UID: "19dade5d-4a01-4adb-af7b-12bb211ef919"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:25:19.252905 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.252881    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/19dade5d-4a01-4adb-af7b-12bb211ef919-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "19dade5d-4a01-4adb-af7b-12bb211ef919" (UID: "19dade5d-4a01-4adb-af7b-12bb211ef919"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 23:25:19.252954 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.252938    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19dade5d-4a01-4adb-af7b-12bb211ef919-istio-token" (OuterVolumeSpecName: "istio-token") pod "19dade5d-4a01-4adb-af7b-12bb211ef919" (UID: "19dade5d-4a01-4adb-af7b-12bb211ef919"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:25:19.351874 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.351765    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-istio-envoy\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:25:19.351874 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.351815    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19dade5d-4a01-4adb-af7b-12bb211ef919-istiod-ca-cert\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:25:19.351874 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.351825    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-workload-socket\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:25:19.351874 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.351834    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19dade5d-4a01-4adb-af7b-12bb211ef919-istio-data\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:25:19.351874 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.351842    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19dade5d-4a01-4adb-af7b-12bb211ef919-istio-podinfo\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:25:19.351874 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.351849    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19dade5d-4a01-4adb-af7b-12bb211ef919-istio-token\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:25:19.351874 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.351859    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djh9t\" (UniqueName: \"kubernetes.io/projected/19dade5d-4a01-4adb-af7b-12bb211ef919-kube-api-access-djh9t\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:25:19.996263 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.996219    2563 generic.go:356] "Generic (PLEG): container finished" podID="19dade5d-4a01-4adb-af7b-12bb211ef919" containerID="5d1e4867a89b5b1a5fc9cba6d1d398586dfd2dbf4c732338708f12d9886b6c5b" exitCode=0
Jul 10 23:25:19.996796 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.996274    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"
Jul 10 23:25:19.996796 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.996310    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb" event={"ID":"19dade5d-4a01-4adb-af7b-12bb211ef919","Type":"ContainerDied","Data":"5d1e4867a89b5b1a5fc9cba6d1d398586dfd2dbf4c732338708f12d9886b6c5b"}
Jul 10 23:25:19.996796 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.996382    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb" event={"ID":"19dade5d-4a01-4adb-af7b-12bb211ef919","Type":"ContainerDied","Data":"3bcac1f508e2023be6b32e6eb2097830e85b921f143bf8d475ac22d57d6537b6"}
Jul 10 23:25:19.996796 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:19.996407    2563 scope.go:117] "RemoveContainer" containerID="5d1e4867a89b5b1a5fc9cba6d1d398586dfd2dbf4c732338708f12d9886b6c5b"
Jul 10 23:25:20.005928 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:20.005904    2563 scope.go:117] "RemoveContainer" containerID="5d1e4867a89b5b1a5fc9cba6d1d398586dfd2dbf4c732338708f12d9886b6c5b"
Jul 10 23:25:20.006264 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:25:20.006241    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d1e4867a89b5b1a5fc9cba6d1d398586dfd2dbf4c732338708f12d9886b6c5b\": container with ID starting with 5d1e4867a89b5b1a5fc9cba6d1d398586dfd2dbf4c732338708f12d9886b6c5b not found: ID does not exist" containerID="5d1e4867a89b5b1a5fc9cba6d1d398586dfd2dbf4c732338708f12d9886b6c5b"
Jul 10 23:25:20.006320 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:20.006274    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d1e4867a89b5b1a5fc9cba6d1d398586dfd2dbf4c732338708f12d9886b6c5b"} err="failed to get container status \"5d1e4867a89b5b1a5fc9cba6d1d398586dfd2dbf4c732338708f12d9886b6c5b\": rpc error: code = NotFound desc = could not find container \"5d1e4867a89b5b1a5fc9cba6d1d398586dfd2dbf4c732338708f12d9886b6c5b\": container with ID starting with 5d1e4867a89b5b1a5fc9cba6d1d398586dfd2dbf4c732338708f12d9886b6c5b not found: ID does not exist"
Jul 10 23:25:20.013763 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:20.013730    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"]
Jul 10 23:25:20.017352 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:20.017321    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-76896fa5-openshift-default-67f795ccc5-2r6nb"]
Jul 10 23:25:20.673028 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:20.672997    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:21.556666 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:21.556630    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19dade5d-4a01-4adb-af7b-12bb211ef919" path="/var/lib/kubelet/pods/19dade5d-4a01-4adb-af7b-12bb211ef919/volumes"
Jul 10 23:25:24.633116 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:24.633088    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:24.693141 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:24.693099    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:24.794992 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:24.794958    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"]
Jul 10 23:25:24.795565 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:24.795540    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19dade5d-4a01-4adb-af7b-12bb211ef919" containerName="istio-proxy"
Jul 10 23:25:24.795565 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:24.795563    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="19dade5d-4a01-4adb-af7b-12bb211ef919" containerName="istio-proxy"
Jul 10 23:25:24.795745 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:24.795630    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="19dade5d-4a01-4adb-af7b-12bb211ef919" containerName="istio-proxy"
Jul 10 23:25:24.801856 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:24.801828    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:24.804445 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:24.804420    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-23bee0fd-openshift-default-dockercfg-nftlz\""
Jul 10 23:25:24.804577 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:24.804417    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 10 23:25:24.812116 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:24.812087    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"]
Jul 10 23:25:24.830934 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:24.830899    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:24.886782 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:24.886695    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:24.902031 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:24.901996    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-workload-certs\") pod \"e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") " pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:24.902163 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:24.902035    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f11e436-0606-46a5-aea7-02dfddf93ccb-istiod-ca-cert\") pod \"e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") " pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:24.902219 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:24.902154    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-workload-socket\") pod \"e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") " pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:24.902259 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:24.902218    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f11e436-0606-46a5-aea7-02dfddf93ccb-istio-token\") pod \"e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") " pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:24.902259 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:24.902239    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-istio-envoy\") pod \"e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") " pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:24.902409 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:24.902267    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f11e436-0606-46a5-aea7-02dfddf93ccb-istio-podinfo\") pod \"e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") " pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:24.902409 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:24.902309    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-credential-socket\") pod \"e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") " pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:24.902409 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:24.902342    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttsjj\" (UniqueName: \"kubernetes.io/projected/5f11e436-0606-46a5-aea7-02dfddf93ccb-kube-api-access-ttsjj\") pod \"e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") " pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:24.902552 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:24.902433    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-istio-data\") pod \"e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") " pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:25.003058 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:25.003020    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f11e436-0606-46a5-aea7-02dfddf93ccb-istio-token\") pod \"e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") " pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:25.003058 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:25.003061    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-istio-envoy\") pod \"e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") " pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:25.003287 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:25.003081    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f11e436-0606-46a5-aea7-02dfddf93ccb-istio-podinfo\") pod \"e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") " pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:25.003287 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:25.003199    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-credential-socket\") pod \"e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") " pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:25.003287 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:25.003239    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttsjj\" (UniqueName: \"kubernetes.io/projected/5f11e436-0606-46a5-aea7-02dfddf93ccb-kube-api-access-ttsjj\") pod \"e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") " pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:25.003287 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:25.003282    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-istio-data\") pod \"e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") " pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:25.003532 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:25.003330    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-workload-certs\") pod \"e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") " pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:25.003532 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:25.003354    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f11e436-0606-46a5-aea7-02dfddf93ccb-istiod-ca-cert\") pod \"e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") " pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:25.003532 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:25.003463    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-workload-socket\") pod \"e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") " pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:25.003687 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:25.003620    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-credential-socket\") pod \"e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") " pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:25.003687 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:25.003665    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-istio-data\") pod \"e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") " pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:25.003797 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:25.003744    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-workload-certs\") pod \"e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") " pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:25.003840 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:25.003811    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-workload-socket\") pod \"e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") " pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:25.004182 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:25.004163    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f11e436-0606-46a5-aea7-02dfddf93ccb-istiod-ca-cert\") pod \"e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") " pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:25.005601 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:25.005575    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-istio-envoy\") pod \"e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") " pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:25.005713 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:25.005694    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f11e436-0606-46a5-aea7-02dfddf93ccb-istio-podinfo\") pod \"e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") " pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:25.010998 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:25.010966    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttsjj\" (UniqueName: \"kubernetes.io/projected/5f11e436-0606-46a5-aea7-02dfddf93ccb-kube-api-access-ttsjj\") pod \"e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") " pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:25.011114 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:25.010971    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f11e436-0606-46a5-aea7-02dfddf93ccb-istio-token\") pod \"e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") " pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:25.114332 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:25.114288    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:25.258339 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:25.258310    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"]
Jul 10 23:25:25.260565 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:25:25.260532    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f11e436_0606_46a5_aea7_02dfddf93ccb.slice/crio-4171fa054f292cf916774e33679c0c3878ac54742b076ce0bb24b24de9e31873 WatchSource:0}: Error finding container 4171fa054f292cf916774e33679c0c3878ac54742b076ce0bb24b24de9e31873: Status 404 returned error can't find the container with id 4171fa054f292cf916774e33679c0c3878ac54742b076ce0bb24b24de9e31873
Jul 10 23:25:25.262782 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:25.262752    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:25:25.262885 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:25.262816    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:25:25.262885 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:25.262844    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:25:25.935948 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:25.935891    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:26.023981 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:26.023954    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:26.029113 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:26.029082    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm" event={"ID":"5f11e436-0606-46a5-aea7-02dfddf93ccb","Type":"ContainerStarted","Data":"1290f8f6c386ffa8a541ed2bfe878f013bfd35f04e36bd6ec1b382b19c064d0b"}
Jul 10 23:25:26.029283 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:26.029143    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm" event={"ID":"5f11e436-0606-46a5-aea7-02dfddf93ccb","Type":"ContainerStarted","Data":"4171fa054f292cf916774e33679c0c3878ac54742b076ce0bb24b24de9e31873"}
Jul 10 23:25:26.057296 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:26.057244    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm" podStartSLOduration=2.057227042 podStartE2EDuration="2.057227042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:25:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:25:26.055123309 +0000 UTC m=+1786.941859452" watchObservedRunningTime="2026-07-10 23:25:26.057227042 +0000 UTC m=+1786.943963184"
Jul 10 23:25:26.087192 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:26.087155    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:26.115279 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:26.115253    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:26.120910 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:26.120880    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:26.269450 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:26.269384    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:26.543667 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:26.543555    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:26.645617 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:26.645578    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:27.034948 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:27.034922    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:27.036255 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:27.036234    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:28.185514 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:28.185467    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:30.376150 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:30.376062    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:30.537909 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:30.537872    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"]
Jul 10 23:25:30.538100 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:30.538058    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm" podUID="5f11e436-0606-46a5-aea7-02dfddf93ccb" containerName="istio-proxy" containerID="cri-o://1290f8f6c386ffa8a541ed2bfe878f013bfd35f04e36bd6ec1b382b19c064d0b" gracePeriod=30
Jul 10 23:25:30.573145 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:30.573108    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:30.653173 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:30.653062    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:30.887077 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:30.887041    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:31.896058 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:31.896030    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:35.800315 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:35.800237    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:35.907884 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:35.907841    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-workload-certs\") pod \"5f11e436-0606-46a5-aea7-02dfddf93ccb\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") "
Jul 10 23:25:35.908074 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:35.907922    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-workload-socket\") pod \"5f11e436-0606-46a5-aea7-02dfddf93ccb\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") "
Jul 10 23:25:35.908074 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:35.907971    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f11e436-0606-46a5-aea7-02dfddf93ccb-istiod-ca-cert\") pod \"5f11e436-0606-46a5-aea7-02dfddf93ccb\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") "
Jul 10 23:25:35.908074 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:35.907997    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f11e436-0606-46a5-aea7-02dfddf93ccb-istio-podinfo\") pod \"5f11e436-0606-46a5-aea7-02dfddf93ccb\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") "
Jul 10 23:25:35.908074 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:35.908033    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-istio-envoy\") pod \"5f11e436-0606-46a5-aea7-02dfddf93ccb\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") "
Jul 10 23:25:35.908285 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:35.908179    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-credential-socket\") pod \"5f11e436-0606-46a5-aea7-02dfddf93ccb\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") "
Jul 10 23:25:35.908285 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:35.908225    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5f11e436-0606-46a5-aea7-02dfddf93ccb" (UID: "5f11e436-0606-46a5-aea7-02dfddf93ccb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:25:35.908285 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:35.908231    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5f11e436-0606-46a5-aea7-02dfddf93ccb" (UID: "5f11e436-0606-46a5-aea7-02dfddf93ccb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:25:35.908285 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:35.908261    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f11e436-0606-46a5-aea7-02dfddf93ccb-istio-token\") pod \"5f11e436-0606-46a5-aea7-02dfddf93ccb\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") "
Jul 10 23:25:35.908540 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:35.908311    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-istio-data\") pod \"5f11e436-0606-46a5-aea7-02dfddf93ccb\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") "
Jul 10 23:25:35.908540 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:35.908421    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ttsjj\" (UniqueName: \"kubernetes.io/projected/5f11e436-0606-46a5-aea7-02dfddf93ccb-kube-api-access-ttsjj\") pod \"5f11e436-0606-46a5-aea7-02dfddf93ccb\" (UID: \"5f11e436-0606-46a5-aea7-02dfddf93ccb\") "
Jul 10 23:25:35.908540 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:35.908460    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f11e436-0606-46a5-aea7-02dfddf93ccb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5f11e436-0606-46a5-aea7-02dfddf93ccb" (UID: "5f11e436-0606-46a5-aea7-02dfddf93ccb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:25:35.908540 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:35.908485    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5f11e436-0606-46a5-aea7-02dfddf93ccb" (UID: "5f11e436-0606-46a5-aea7-02dfddf93ccb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:25:35.908744 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:35.908696    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-workload-certs\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:25:35.908744 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:35.908703    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-istio-data" (OuterVolumeSpecName: "istio-data") pod "5f11e436-0606-46a5-aea7-02dfddf93ccb" (UID: "5f11e436-0606-46a5-aea7-02dfddf93ccb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:25:35.908744 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:35.908716    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-workload-socket\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:25:35.908744 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:35.908729    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f11e436-0606-46a5-aea7-02dfddf93ccb-istiod-ca-cert\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:25:35.908744 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:35.908741    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-credential-socket\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:25:35.910423 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:35.910397    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5f11e436-0606-46a5-aea7-02dfddf93ccb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5f11e436-0606-46a5-aea7-02dfddf93ccb" (UID: "5f11e436-0606-46a5-aea7-02dfddf93ccb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 23:25:35.910538 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:35.910431    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5f11e436-0606-46a5-aea7-02dfddf93ccb" (UID: "5f11e436-0606-46a5-aea7-02dfddf93ccb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:25:35.910538 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:35.910470    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f11e436-0606-46a5-aea7-02dfddf93ccb-istio-token" (OuterVolumeSpecName: "istio-token") pod "5f11e436-0606-46a5-aea7-02dfddf93ccb" (UID: "5f11e436-0606-46a5-aea7-02dfddf93ccb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:25:35.910620 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:35.910601    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f11e436-0606-46a5-aea7-02dfddf93ccb-kube-api-access-ttsjj" (OuterVolumeSpecName: "kube-api-access-ttsjj") pod "5f11e436-0606-46a5-aea7-02dfddf93ccb" (UID: "5f11e436-0606-46a5-aea7-02dfddf93ccb"). InnerVolumeSpecName "kube-api-access-ttsjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:25:36.010026 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:36.009980    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f11e436-0606-46a5-aea7-02dfddf93ccb-istio-token\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:25:36.010026 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:36.010012    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-istio-data\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:25:36.010026 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:36.010023    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttsjj\" (UniqueName: \"kubernetes.io/projected/5f11e436-0606-46a5-aea7-02dfddf93ccb-kube-api-access-ttsjj\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:25:36.010026 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:36.010033    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f11e436-0606-46a5-aea7-02dfddf93ccb-istio-podinfo\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:25:36.010026 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:36.010042    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f11e436-0606-46a5-aea7-02dfddf93ccb-istio-envoy\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:25:36.079391 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:36.079274    2563 generic.go:356] "Generic (PLEG): container finished" podID="5f11e436-0606-46a5-aea7-02dfddf93ccb" containerID="1290f8f6c386ffa8a541ed2bfe878f013bfd35f04e36bd6ec1b382b19c064d0b" exitCode=0
Jul 10 23:25:36.079391 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:36.079326    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"
Jul 10 23:25:36.079391 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:36.079375    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm" event={"ID":"5f11e436-0606-46a5-aea7-02dfddf93ccb","Type":"ContainerDied","Data":"1290f8f6c386ffa8a541ed2bfe878f013bfd35f04e36bd6ec1b382b19c064d0b"}
Jul 10 23:25:36.079616 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:36.079438    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm" event={"ID":"5f11e436-0606-46a5-aea7-02dfddf93ccb","Type":"ContainerDied","Data":"4171fa054f292cf916774e33679c0c3878ac54742b076ce0bb24b24de9e31873"}
Jul 10 23:25:36.079616 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:36.079465    2563 scope.go:117] "RemoveContainer" containerID="1290f8f6c386ffa8a541ed2bfe878f013bfd35f04e36bd6ec1b382b19c064d0b"
Jul 10 23:25:36.089720 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:36.089695    2563 scope.go:117] "RemoveContainer" containerID="1290f8f6c386ffa8a541ed2bfe878f013bfd35f04e36bd6ec1b382b19c064d0b"
Jul 10 23:25:36.090025 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:25:36.090007    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1290f8f6c386ffa8a541ed2bfe878f013bfd35f04e36bd6ec1b382b19c064d0b\": container with ID starting with 1290f8f6c386ffa8a541ed2bfe878f013bfd35f04e36bd6ec1b382b19c064d0b not found: ID does not exist" containerID="1290f8f6c386ffa8a541ed2bfe878f013bfd35f04e36bd6ec1b382b19c064d0b"
Jul 10 23:25:36.090066 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:36.090039    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1290f8f6c386ffa8a541ed2bfe878f013bfd35f04e36bd6ec1b382b19c064d0b"} err="failed to get container status \"1290f8f6c386ffa8a541ed2bfe878f013bfd35f04e36bd6ec1b382b19c064d0b\": rpc error: code = NotFound desc = could not find container \"1290f8f6c386ffa8a541ed2bfe878f013bfd35f04e36bd6ec1b382b19c064d0b\": container with ID starting with 1290f8f6c386ffa8a541ed2bfe878f013bfd35f04e36bd6ec1b382b19c064d0b not found: ID does not exist"
Jul 10 23:25:36.101283 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:36.101241    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"]
Jul 10 23:25:36.107756 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:36.107718    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-23bee0fd-openshift-default-78d94c797f-56frm"]
Jul 10 23:25:36.363576 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:36.363477    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:36.463811 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:36.463777    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:36.836016 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:36.835983    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:36.987882 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:36.987850    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:37.084661 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:37.084626    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:37.558721 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:37.558662    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f11e436-0606-46a5-aea7-02dfddf93ccb" path="/var/lib/kubelet/pods/5f11e436-0606-46a5-aea7-02dfddf93ccb/volumes"
Jul 10 23:25:38.461302 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:38.461265    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:39.614417 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:39.614389    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/ovn-acl-logging/0.log"
Jul 10 23:25:39.616797 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:39.616776    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/ovn-acl-logging/0.log"
Jul 10 23:25:44.038067 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:44.038020    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:44.144984 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:44.144947    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:44.283423 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:44.283391    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:44.383469 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:44.383345    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:25:44.772487 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:25:44.772452    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:26:10.766653 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:26:10.766617    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:26:16.387767 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:26:16.387730    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:26:16.447851 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:26:16.447822    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:26:16.517270 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:26:16.517239    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:26:16.616028 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:26:16.615997    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:26:32.489161 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:26:32.489124    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:26:37.988008 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:26:37.987973    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:26:38.048264 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:26:38.048229    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:26:38.120623 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:26:38.120589    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:26:38.248443 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:26:38.248329    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:26:48.748376 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:26:48.748329    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:26:49.546185 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:26:49.546150    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:26:55.144553 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:26:55.144513    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:26:55.250752 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:26:55.250719    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:26:55.345210 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:26:55.345178    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:26:55.444906 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:26:55.444817    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:26:56.236722 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:26:56.236681    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:26:56.337266 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:26:56.337233    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:26:56.436443 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:26:56.436412    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:26:58.150410 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:26:58.150349    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:03.801960 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:03.801878    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:03.859349 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:03.859302    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:04.473402 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:04.473319    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:10.106241 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.106208    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:10.205624 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.205593    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:10.345104 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.345061    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:10.710742 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.710696    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"]
Jul 10 23:27:10.711259 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.711244    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f11e436-0606-46a5-aea7-02dfddf93ccb" containerName="istio-proxy"
Jul 10 23:27:10.711305 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.711263    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f11e436-0606-46a5-aea7-02dfddf93ccb" containerName="istio-proxy"
Jul 10 23:27:10.711410 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.711398    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f11e436-0606-46a5-aea7-02dfddf93ccb" containerName="istio-proxy"
Jul 10 23:27:10.714689 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.714668    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:10.717796 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.717773    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-62569fb2-openshift-default-dockercfg-pg5n7\""
Jul 10 23:27:10.717796 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.717788    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 10 23:27:10.729555 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.729528    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:10.738608 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.738575    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"]
Jul 10 23:27:10.830067 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.830037    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:10.848288 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.848253    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-credential-socket\") pod \"e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") " pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:10.848473 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.848317    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9567654f-156b-4c9c-81b6-d49c33f32a69-istio-token\") pod \"e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") " pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:10.848473 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.848342    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-istio-data\") pod \"e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") " pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:10.848473 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.848402    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-workload-socket\") pod \"e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") " pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:10.848652 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.848484    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pg2b\" (UniqueName: \"kubernetes.io/projected/9567654f-156b-4c9c-81b6-d49c33f32a69-kube-api-access-7pg2b\") pod \"e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") " pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:10.848652 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.848527    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-workload-certs\") pod \"e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") " pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:10.848652 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.848557    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9567654f-156b-4c9c-81b6-d49c33f32a69-istiod-ca-cert\") pod \"e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") " pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:10.848652 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.848594    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-istio-envoy\") pod \"e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") " pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:10.848652 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.848634    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9567654f-156b-4c9c-81b6-d49c33f32a69-istio-podinfo\") pod \"e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") " pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:10.950112 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.950072    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9567654f-156b-4c9c-81b6-d49c33f32a69-istio-token\") pod \"e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") " pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:10.950112 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.950116    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-istio-data\") pod \"e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") " pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:10.950344 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.950245    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-workload-socket\") pod \"e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") " pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:10.950344 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.950309    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pg2b\" (UniqueName: \"kubernetes.io/projected/9567654f-156b-4c9c-81b6-d49c33f32a69-kube-api-access-7pg2b\") pod \"e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") " pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:10.950344 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.950340    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-workload-certs\") pod \"e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") " pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:10.950546 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.950399    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9567654f-156b-4c9c-81b6-d49c33f32a69-istiod-ca-cert\") pod \"e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") " pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:10.950546 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.950440    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-istio-envoy\") pod \"e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") " pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:10.950546 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.950484    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9567654f-156b-4c9c-81b6-d49c33f32a69-istio-podinfo\") pod \"e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") " pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:10.950546 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.950521    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-credential-socket\") pod \"e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") " pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:10.950741 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.950577    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-istio-data\") pod \"e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") " pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:10.950741 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.950667    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-workload-socket\") pod \"e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") " pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:10.950807 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.950759    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-workload-certs\") pod \"e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") " pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:10.950842 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.950829    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-credential-socket\") pod \"e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") " pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:10.951169 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.951146    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9567654f-156b-4c9c-81b6-d49c33f32a69-istiod-ca-cert\") pod \"e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") " pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:10.953012 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.952984    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-istio-envoy\") pod \"e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") " pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:10.953158 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.953141    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9567654f-156b-4c9c-81b6-d49c33f32a69-istio-podinfo\") pod \"e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") " pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:10.958517 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.958489    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pg2b\" (UniqueName: \"kubernetes.io/projected/9567654f-156b-4c9c-81b6-d49c33f32a69-kube-api-access-7pg2b\") pod \"e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") " pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:10.958630 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:10.958606    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9567654f-156b-4c9c-81b6-d49c33f32a69-istio-token\") pod \"e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") " pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:11.025330 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:11.025287    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:11.171798 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:11.171770    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"]
Jul 10 23:27:11.174407 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:27:11.174345    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9567654f_156b_4c9c_81b6_d49c33f32a69.slice/crio-785f7605f7453b8b9b39d287568cbdf79c96109d68c97bad87347ed7f1e02dc9 WatchSource:0}: Error finding container 785f7605f7453b8b9b39d287568cbdf79c96109d68c97bad87347ed7f1e02dc9: Status 404 returned error can't find the container with id 785f7605f7453b8b9b39d287568cbdf79c96109d68c97bad87347ed7f1e02dc9
Jul 10 23:27:11.177014 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:11.176972    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:27:11.177083 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:11.177066    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:27:11.177131 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:11.177115    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:27:11.562282 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:11.556950    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74" event={"ID":"9567654f-156b-4c9c-81b6-d49c33f32a69","Type":"ContainerStarted","Data":"288f8fa69bab80d0cc59fa20cfbb3c7378a3ea35b77f4ada49a15d937559b352"}
Jul 10 23:27:11.562282 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:11.557030    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74" event={"ID":"9567654f-156b-4c9c-81b6-d49c33f32a69","Type":"ContainerStarted","Data":"785f7605f7453b8b9b39d287568cbdf79c96109d68c97bad87347ed7f1e02dc9"}
Jul 10 23:27:11.578845 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:11.578782    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74" podStartSLOduration=1.5787614269999999 podStartE2EDuration="1.578761427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:27:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:27:11.576802453 +0000 UTC m=+1892.463538632" watchObservedRunningTime="2026-07-10 23:27:11.578761427 +0000 UTC m=+1892.465497569"
Jul 10 23:27:12.025827 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:12.025788    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:12.031019 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:12.030987    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:12.557355 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:12.557320    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:12.558355 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:12.558333    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:13.469337 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:13.469302    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:13.542523 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:13.542490    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:13.604161 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:13.604125    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:14.038608 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:14.038573    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:14.139355 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:14.139323    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:14.241397 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:14.241329    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:17.971870 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:17.971841    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:18.615503 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:18.615467    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:18.723694 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:18.723656    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:19.063064 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:19.063030    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:24.891703 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:24.891666    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:24.954857 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:24.954824    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:25.319656 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:25.319619    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:25.762488 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:25.762452    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:25.865585 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:25.865547    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:27.787808 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:27.786706    2563 patch_prober.go:28] interesting pod/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": read tcp 10.132.0.2:50896->10.132.0.42:15021: read: connection reset by peer" start-of-body=
Jul 10 23:27:27.787808 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:27.786775    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74" podUID="9567654f-156b-4c9c-81b6-d49c33f32a69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": read tcp 10.132.0.2:50896->10.132.0.42:15021: read: connection reset by peer"
Jul 10 23:27:28.636105 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:28.636072    2563 generic.go:356] "Generic (PLEG): container finished" podID="9567654f-156b-4c9c-81b6-d49c33f32a69" containerID="288f8fa69bab80d0cc59fa20cfbb3c7378a3ea35b77f4ada49a15d937559b352" exitCode=0
Jul 10 23:27:28.636321 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:28.636141    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74" event={"ID":"9567654f-156b-4c9c-81b6-d49c33f32a69","Type":"ContainerDied","Data":"288f8fa69bab80d0cc59fa20cfbb3c7378a3ea35b77f4ada49a15d937559b352"}
Jul 10 23:27:28.636634 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:28.636616    2563 scope.go:117] "RemoveContainer" containerID="288f8fa69bab80d0cc59fa20cfbb3c7378a3ea35b77f4ada49a15d937559b352"
Jul 10 23:27:28.637703 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:28.637677    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:27:28.637793 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:28.637757    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:27:28.637860 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:28.637807    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:27:29.645659 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:29.645623    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74" event={"ID":"9567654f-156b-4c9c-81b6-d49c33f32a69","Type":"ContainerStarted","Data":"35312387739b5b0a9f01399a2a2e7c87fa53bec2571c29263a15f36a35145cce"}
Jul 10 23:27:30.025622 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:30.025580    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:30.029412 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:30.029382    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:30.650970 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:30.650940    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:30.651923 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:30.651900    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:31.354186 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:31.354089    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:31.404581 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:31.404545    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:31.589429 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:31.589357    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"]
Jul 10 23:27:31.608720 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:31.608599    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:31.655809 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:31.655775    2563 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74" secret="" err="secret \"e2e-mt-62569fb2-openshift-default-dockercfg-pg5n7\" not found"
Jul 10 23:27:31.835837 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:31.835782    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:31.954505 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:31.954423    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:32.053509 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:32.053477    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:32.164014 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:32.163979    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:32.660799 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:32.660757    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74" podUID="9567654f-156b-4c9c-81b6-d49c33f32a69" containerName="istio-proxy" containerID="cri-o://35312387739b5b0a9f01399a2a2e7c87fa53bec2571c29263a15f36a35145cce" gracePeriod=30
Jul 10 23:27:37.831448 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:37.831408    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:37.889581 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:37.889548    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:37.926552 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:37.926451    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:37.960381 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:37.960302    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:38.005717 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.005678    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9567654f-156b-4c9c-81b6-d49c33f32a69-istiod-ca-cert\") pod \"9567654f-156b-4c9c-81b6-d49c33f32a69\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") "
Jul 10 23:27:38.005717 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.005716    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-istio-envoy\") pod \"9567654f-156b-4c9c-81b6-d49c33f32a69\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") "
Jul 10 23:27:38.005971 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.005737    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-credential-socket\") pod \"9567654f-156b-4c9c-81b6-d49c33f32a69\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") "
Jul 10 23:27:38.005971 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.005770    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-istio-data\") pod \"9567654f-156b-4c9c-81b6-d49c33f32a69\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") "
Jul 10 23:27:38.005971 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.005906    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9567654f-156b-4c9c-81b6-d49c33f32a69-istio-token\") pod \"9567654f-156b-4c9c-81b6-d49c33f32a69\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") "
Jul 10 23:27:38.005971 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.005957    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-workload-socket\") pod \"9567654f-156b-4c9c-81b6-d49c33f32a69\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") "
Jul 10 23:27:38.006194 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.005987    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-workload-certs\") pod \"9567654f-156b-4c9c-81b6-d49c33f32a69\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") "
Jul 10 23:27:38.006194 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.006021    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7pg2b\" (UniqueName: \"kubernetes.io/projected/9567654f-156b-4c9c-81b6-d49c33f32a69-kube-api-access-7pg2b\") pod \"9567654f-156b-4c9c-81b6-d49c33f32a69\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") "
Jul 10 23:27:38.006194 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.006099    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9567654f-156b-4c9c-81b6-d49c33f32a69-istio-podinfo\") pod \"9567654f-156b-4c9c-81b6-d49c33f32a69\" (UID: \"9567654f-156b-4c9c-81b6-d49c33f32a69\") "
Jul 10 23:27:38.006194 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.006108    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9567654f-156b-4c9c-81b6-d49c33f32a69-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9567654f-156b-4c9c-81b6-d49c33f32a69" (UID: "9567654f-156b-4c9c-81b6-d49c33f32a69"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:27:38.006194 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.006143    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9567654f-156b-4c9c-81b6-d49c33f32a69" (UID: "9567654f-156b-4c9c-81b6-d49c33f32a69"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:27:38.006484 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.006267    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9567654f-156b-4c9c-81b6-d49c33f32a69" (UID: "9567654f-156b-4c9c-81b6-d49c33f32a69"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:27:38.006484 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.006324    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9567654f-156b-4c9c-81b6-d49c33f32a69" (UID: "9567654f-156b-4c9c-81b6-d49c33f32a69"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:27:38.006593 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.006546    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9567654f-156b-4c9c-81b6-d49c33f32a69-istiod-ca-cert\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:27:38.006593 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.006575    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-credential-socket\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:27:38.006593 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.006591    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-workload-socket\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:27:38.006702 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.006604    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-workload-certs\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:27:38.006743 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.006717    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-istio-data" (OuterVolumeSpecName: "istio-data") pod "9567654f-156b-4c9c-81b6-d49c33f32a69" (UID: "9567654f-156b-4c9c-81b6-d49c33f32a69"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:27:38.008531 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.008506    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9567654f-156b-4c9c-81b6-d49c33f32a69-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9567654f-156b-4c9c-81b6-d49c33f32a69" (UID: "9567654f-156b-4c9c-81b6-d49c33f32a69"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 23:27:38.008531 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.008524    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9567654f-156b-4c9c-81b6-d49c33f32a69-kube-api-access-7pg2b" (OuterVolumeSpecName: "kube-api-access-7pg2b") pod "9567654f-156b-4c9c-81b6-d49c33f32a69" (UID: "9567654f-156b-4c9c-81b6-d49c33f32a69"). InnerVolumeSpecName "kube-api-access-7pg2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:27:38.008747 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.008727    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9567654f-156b-4c9c-81b6-d49c33f32a69-istio-token" (OuterVolumeSpecName: "istio-token") pod "9567654f-156b-4c9c-81b6-d49c33f32a69" (UID: "9567654f-156b-4c9c-81b6-d49c33f32a69"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:27:38.008807 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.008744    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9567654f-156b-4c9c-81b6-d49c33f32a69" (UID: "9567654f-156b-4c9c-81b6-d49c33f32a69"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:27:38.060145 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.060105    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:38.107113 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.107082    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9567654f-156b-4c9c-81b6-d49c33f32a69-istio-podinfo\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:27:38.107113 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.107109    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-istio-envoy\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:27:38.107113 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.107120    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9567654f-156b-4c9c-81b6-d49c33f32a69-istio-data\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:27:38.107401 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.107130    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9567654f-156b-4c9c-81b6-d49c33f32a69-istio-token\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:27:38.107401 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.107140    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pg2b\" (UniqueName: \"kubernetes.io/projected/9567654f-156b-4c9c-81b6-d49c33f32a69-kube-api-access-7pg2b\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:27:38.178750 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.178654    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:38.690825 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.690785    2563 generic.go:356] "Generic (PLEG): container finished" podID="9567654f-156b-4c9c-81b6-d49c33f32a69" containerID="35312387739b5b0a9f01399a2a2e7c87fa53bec2571c29263a15f36a35145cce" exitCode=0
Jul 10 23:27:38.690825 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.690829    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74" event={"ID":"9567654f-156b-4c9c-81b6-d49c33f32a69","Type":"ContainerDied","Data":"35312387739b5b0a9f01399a2a2e7c87fa53bec2571c29263a15f36a35145cce"}
Jul 10 23:27:38.691067 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.690855    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"
Jul 10 23:27:38.691067 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.690874    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74" event={"ID":"9567654f-156b-4c9c-81b6-d49c33f32a69","Type":"ContainerDied","Data":"785f7605f7453b8b9b39d287568cbdf79c96109d68c97bad87347ed7f1e02dc9"}
Jul 10 23:27:38.691067 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.690896    2563 scope.go:117] "RemoveContainer" containerID="35312387739b5b0a9f01399a2a2e7c87fa53bec2571c29263a15f36a35145cce"
Jul 10 23:27:38.700596 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.700576    2563 scope.go:117] "RemoveContainer" containerID="288f8fa69bab80d0cc59fa20cfbb3c7378a3ea35b77f4ada49a15d937559b352"
Jul 10 23:27:38.709479 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.709455    2563 scope.go:117] "RemoveContainer" containerID="35312387739b5b0a9f01399a2a2e7c87fa53bec2571c29263a15f36a35145cce"
Jul 10 23:27:38.709951 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:27:38.709914    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35312387739b5b0a9f01399a2a2e7c87fa53bec2571c29263a15f36a35145cce\": container with ID starting with 35312387739b5b0a9f01399a2a2e7c87fa53bec2571c29263a15f36a35145cce not found: ID does not exist" containerID="35312387739b5b0a9f01399a2a2e7c87fa53bec2571c29263a15f36a35145cce"
Jul 10 23:27:38.710064 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.709956    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35312387739b5b0a9f01399a2a2e7c87fa53bec2571c29263a15f36a35145cce"} err="failed to get container status \"35312387739b5b0a9f01399a2a2e7c87fa53bec2571c29263a15f36a35145cce\": rpc error: code = NotFound desc = could not find container \"35312387739b5b0a9f01399a2a2e7c87fa53bec2571c29263a15f36a35145cce\": container with ID starting with 35312387739b5b0a9f01399a2a2e7c87fa53bec2571c29263a15f36a35145cce not found: ID does not exist"
Jul 10 23:27:38.710064 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.709981    2563 scope.go:117] "RemoveContainer" containerID="288f8fa69bab80d0cc59fa20cfbb3c7378a3ea35b77f4ada49a15d937559b352"
Jul 10 23:27:38.710354 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:27:38.710310    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"288f8fa69bab80d0cc59fa20cfbb3c7378a3ea35b77f4ada49a15d937559b352\": container with ID starting with 288f8fa69bab80d0cc59fa20cfbb3c7378a3ea35b77f4ada49a15d937559b352 not found: ID does not exist" containerID="288f8fa69bab80d0cc59fa20cfbb3c7378a3ea35b77f4ada49a15d937559b352"
Jul 10 23:27:38.710498 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.710385    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"288f8fa69bab80d0cc59fa20cfbb3c7378a3ea35b77f4ada49a15d937559b352"} err="failed to get container status \"288f8fa69bab80d0cc59fa20cfbb3c7378a3ea35b77f4ada49a15d937559b352\": rpc error: code = NotFound desc = could not find container \"288f8fa69bab80d0cc59fa20cfbb3c7378a3ea35b77f4ada49a15d937559b352\": container with ID starting with 288f8fa69bab80d0cc59fa20cfbb3c7378a3ea35b77f4ada49a15d937559b352 not found: ID does not exist"
Jul 10 23:27:38.712226 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.712204    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"]
Jul 10 23:27:38.718104 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:38.718059    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-62569fb2-openshift-default-b655bc4f9-wjx74"]
Jul 10 23:27:39.556393 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:39.556338    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9567654f-156b-4c9c-81b6-d49c33f32a69" path="/var/lib/kubelet/pods/9567654f-156b-4c9c-81b6-d49c33f32a69/volumes"
Jul 10 23:27:43.686573 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:43.686536    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:43.739547 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:43.739512    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:50.450621 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.450585    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:50.719252 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.719162    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"]
Jul 10 23:27:50.719584 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.719570    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9567654f-156b-4c9c-81b6-d49c33f32a69" containerName="istio-proxy"
Jul 10 23:27:50.719655 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.719586    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9567654f-156b-4c9c-81b6-d49c33f32a69" containerName="istio-proxy"
Jul 10 23:27:50.719655 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.719596    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9567654f-156b-4c9c-81b6-d49c33f32a69" containerName="istio-proxy"
Jul 10 23:27:50.719655 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.719602    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9567654f-156b-4c9c-81b6-d49c33f32a69" containerName="istio-proxy"
Jul 10 23:27:50.719815 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.719664    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9567654f-156b-4c9c-81b6-d49c33f32a69" containerName="istio-proxy"
Jul 10 23:27:50.722784 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.722763    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:50.724972 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.724947    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7e40d510-openshift-default-dockercfg-pccr7\""
Jul 10 23:27:50.725105 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.725088    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 10 23:27:50.734590 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.734564    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"]
Jul 10 23:27:50.747083 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.747045    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:50.828796 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.828752    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-credential-socket\") pod \"e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") " pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:50.828796 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.828794    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-istio-data\") pod \"e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") " pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:50.829044 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.828824    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-istio-envoy\") pod \"e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") " pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:50.829044 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.828898    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba213e27-81b4-4059-b274-a895caac972c-istio-token\") pod \"e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") " pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:50.829044 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.828974    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-workload-certs\") pod \"e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") " pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:50.829044 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.829027    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba213e27-81b4-4059-b274-a895caac972c-istiod-ca-cert\") pod \"e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") " pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:50.829200 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.829062    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-workload-socket\") pod \"e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") " pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:50.829200 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.829097    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7jf6\" (UniqueName: \"kubernetes.io/projected/ba213e27-81b4-4059-b274-a895caac972c-kube-api-access-j7jf6\") pod \"e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") " pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:50.829200 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.829154    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba213e27-81b4-4059-b274-a895caac972c-istio-podinfo\") pod \"e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") " pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:50.930558 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.930514    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-istio-envoy\") pod \"e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") " pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:50.930558 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.930560    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba213e27-81b4-4059-b274-a895caac972c-istio-token\") pod \"e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") " pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:50.930825 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.930609    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-workload-certs\") pod \"e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") " pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:50.930825 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.930635    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba213e27-81b4-4059-b274-a895caac972c-istiod-ca-cert\") pod \"e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") " pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:50.930825 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.930666    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-workload-socket\") pod \"e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") " pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:50.930825 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.930698    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7jf6\" (UniqueName: \"kubernetes.io/projected/ba213e27-81b4-4059-b274-a895caac972c-kube-api-access-j7jf6\") pod \"e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") " pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:50.930825 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.930744    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba213e27-81b4-4059-b274-a895caac972c-istio-podinfo\") pod \"e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") " pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:50.930825 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.930778    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-credential-socket\") pod \"e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") " pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:50.930825 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.930799    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-istio-data\") pod \"e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") " pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:50.931185 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.931091    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-workload-socket\") pod \"e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") " pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:50.931185 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.931172    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-istio-data\") pod \"e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") " pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:50.931299 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.931250    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-credential-socket\") pod \"e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") " pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:50.931494 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.931475    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba213e27-81b4-4059-b274-a895caac972c-istiod-ca-cert\") pod \"e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") " pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:50.931569 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.931503    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-workload-certs\") pod \"e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") " pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:50.933107 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.933077    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-istio-envoy\") pod \"e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") " pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:50.933542 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.933520    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba213e27-81b4-4059-b274-a895caac972c-istio-podinfo\") pod \"e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") " pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:50.938951 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.938919    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba213e27-81b4-4059-b274-a895caac972c-istio-token\") pod \"e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") " pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:50.939107 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:50.939081    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7jf6\" (UniqueName: \"kubernetes.io/projected/ba213e27-81b4-4059-b274-a895caac972c-kube-api-access-j7jf6\") pod \"e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") " pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:51.034924 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:51.034881    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:51.191447 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:51.191420    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"]
Jul 10 23:27:51.193787 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:27:51.193754    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba213e27_81b4_4059_b274_a895caac972c.slice/crio-4be2014b432329634e1e762ca20d76235f76e7e69819b7b6642f0dc209eaca28 WatchSource:0}: Error finding container 4be2014b432329634e1e762ca20d76235f76e7e69819b7b6642f0dc209eaca28: Status 404 returned error can't find the container with id 4be2014b432329634e1e762ca20d76235f76e7e69819b7b6642f0dc209eaca28
Jul 10 23:27:51.196027 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:51.195991    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:27:51.196131 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:51.196064    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:27:51.196131 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:51.196090    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:27:51.224014 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:51.223983    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:51.756863 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:51.756827    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8" event={"ID":"ba213e27-81b4-4059-b274-a895caac972c","Type":"ContainerStarted","Data":"4a4652528b28abf1b458bcb2e701c31cdb26c4a28153a96da93ba3b7f93bf619"}
Jul 10 23:27:51.757269 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:51.756882    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8" event={"ID":"ba213e27-81b4-4059-b274-a895caac972c","Type":"ContainerStarted","Data":"4be2014b432329634e1e762ca20d76235f76e7e69819b7b6642f0dc209eaca28"}
Jul 10 23:27:51.777594 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:51.777542    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8" podStartSLOduration=1.777524774 podStartE2EDuration="1.777524774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:27:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:27:51.775032735 +0000 UTC m=+1932.661768898" watchObservedRunningTime="2026-07-10 23:27:51.777524774 +0000 UTC m=+1932.664260916"
Jul 10 23:27:52.035788 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:52.035694    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:52.041281 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:52.041252    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:52.762009 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:52.761977    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:52.762928 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:52.762909    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:27:53.690390 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:53.690330    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:53.766435 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:53.766384    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:53.824091 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:53.824062    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:54.170001 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:54.169967    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:54.270018 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:54.269984    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:54.370352 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:54.370318    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:27:58.147582 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:27:58.147544    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:28:07.909469 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:07.908377    2563 patch_prober.go:28] interesting pod/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": read tcp 10.132.0.2:55268->10.132.0.43:15021: read: connection reset by peer" start-of-body=
Jul 10 23:28:07.909469 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:07.908454    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8" podUID="ba213e27-81b4-4059-b274-a895caac972c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": read tcp 10.132.0.2:55268->10.132.0.43:15021: read: connection reset by peer"
Jul 10 23:28:08.856700 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:08.856664    2563 generic.go:356] "Generic (PLEG): container finished" podID="ba213e27-81b4-4059-b274-a895caac972c" containerID="4a4652528b28abf1b458bcb2e701c31cdb26c4a28153a96da93ba3b7f93bf619" exitCode=0
Jul 10 23:28:08.857098 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:08.856733    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8" event={"ID":"ba213e27-81b4-4059-b274-a895caac972c","Type":"ContainerDied","Data":"4a4652528b28abf1b458bcb2e701c31cdb26c4a28153a96da93ba3b7f93bf619"}
Jul 10 23:28:08.857199 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:08.857181    2563 scope.go:117] "RemoveContainer" containerID="4a4652528b28abf1b458bcb2e701c31cdb26c4a28153a96da93ba3b7f93bf619"
Jul 10 23:28:08.858490 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:08.858455    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:28:08.858570 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:08.858552    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:28:08.858623 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:08.858606    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:28:09.314469 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.314430    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"]
Jul 10 23:28:09.315076 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.315053    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9567654f-156b-4c9c-81b6-d49c33f32a69" containerName="istio-proxy"
Jul 10 23:28:09.320078 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.320053    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"
Jul 10 23:28:09.323329 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.322867    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7e40d510\"/\"kube-root-ca.crt\""
Jul 10 23:28:09.323329 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.323090    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7e40d510\"/\"openshift-service-ca.crt\""
Jul 10 23:28:09.324080 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.323725    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7e40d510\"/\"default-dockercfg-7z2nq\""
Jul 10 23:28:09.324080 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.323924    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7e40d510\"/\"e2e-lifecycle-model-7e40d510-kserve-self-signed-certs\""
Jul 10 23:28:09.325331 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.325305    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"]
Jul 10 23:28:09.341994 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.341959    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:28:09.420290 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.420247    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84608566-23d5-44cd-bf78-b13e83008894-tls-certs\") pod \"e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv\" (UID: \"84608566-23d5-44cd-bf78-b13e83008894\") " pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"
Jul 10 23:28:09.420523 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.420307    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84608566-23d5-44cd-bf78-b13e83008894-home\") pod \"e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv\" (UID: \"84608566-23d5-44cd-bf78-b13e83008894\") " pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"
Jul 10 23:28:09.420523 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.420409    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84608566-23d5-44cd-bf78-b13e83008894-tmp-dir\") pod \"e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv\" (UID: \"84608566-23d5-44cd-bf78-b13e83008894\") " pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"
Jul 10 23:28:09.420523 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.420478    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr6wd\" (UniqueName: \"kubernetes.io/projected/84608566-23d5-44cd-bf78-b13e83008894-kube-api-access-cr6wd\") pod \"e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv\" (UID: \"84608566-23d5-44cd-bf78-b13e83008894\") " pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"
Jul 10 23:28:09.420523 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.420499    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84608566-23d5-44cd-bf78-b13e83008894-model-cache\") pod \"e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv\" (UID: \"84608566-23d5-44cd-bf78-b13e83008894\") " pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"
Jul 10 23:28:09.420523 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.420520    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84608566-23d5-44cd-bf78-b13e83008894-dshm\") pod \"e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv\" (UID: \"84608566-23d5-44cd-bf78-b13e83008894\") " pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"
Jul 10 23:28:09.521893 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.521850    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr6wd\" (UniqueName: \"kubernetes.io/projected/84608566-23d5-44cd-bf78-b13e83008894-kube-api-access-cr6wd\") pod \"e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv\" (UID: \"84608566-23d5-44cd-bf78-b13e83008894\") " pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"
Jul 10 23:28:09.521893 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.521892    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84608566-23d5-44cd-bf78-b13e83008894-model-cache\") pod \"e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv\" (UID: \"84608566-23d5-44cd-bf78-b13e83008894\") " pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"
Jul 10 23:28:09.522173 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.521912    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84608566-23d5-44cd-bf78-b13e83008894-dshm\") pod \"e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv\" (UID: \"84608566-23d5-44cd-bf78-b13e83008894\") " pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"
Jul 10 23:28:09.522173 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.521937    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84608566-23d5-44cd-bf78-b13e83008894-tls-certs\") pod \"e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv\" (UID: \"84608566-23d5-44cd-bf78-b13e83008894\") " pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"
Jul 10 23:28:09.522173 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.522028    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84608566-23d5-44cd-bf78-b13e83008894-home\") pod \"e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv\" (UID: \"84608566-23d5-44cd-bf78-b13e83008894\") " pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"
Jul 10 23:28:09.522173 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.522115    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84608566-23d5-44cd-bf78-b13e83008894-tmp-dir\") pod \"e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv\" (UID: \"84608566-23d5-44cd-bf78-b13e83008894\") " pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"
Jul 10 23:28:09.522453 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.522430    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84608566-23d5-44cd-bf78-b13e83008894-home\") pod \"e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv\" (UID: \"84608566-23d5-44cd-bf78-b13e83008894\") " pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"
Jul 10 23:28:09.522520 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.522487    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84608566-23d5-44cd-bf78-b13e83008894-model-cache\") pod \"e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv\" (UID: \"84608566-23d5-44cd-bf78-b13e83008894\") " pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"
Jul 10 23:28:09.522520 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.522511    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84608566-23d5-44cd-bf78-b13e83008894-tmp-dir\") pod \"e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv\" (UID: \"84608566-23d5-44cd-bf78-b13e83008894\") " pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"
Jul 10 23:28:09.524553 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.524529    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84608566-23d5-44cd-bf78-b13e83008894-dshm\") pod \"e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv\" (UID: \"84608566-23d5-44cd-bf78-b13e83008894\") " pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"
Jul 10 23:28:09.524745 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.524729    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84608566-23d5-44cd-bf78-b13e83008894-tls-certs\") pod \"e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv\" (UID: \"84608566-23d5-44cd-bf78-b13e83008894\") " pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"
Jul 10 23:28:09.530158 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.530133    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr6wd\" (UniqueName: \"kubernetes.io/projected/84608566-23d5-44cd-bf78-b13e83008894-kube-api-access-cr6wd\") pod \"e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv\" (UID: \"84608566-23d5-44cd-bf78-b13e83008894\") " pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"
Jul 10 23:28:09.631427 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.631312    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"
Jul 10 23:28:09.816240 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.816211    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"]
Jul 10 23:28:09.818305 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:28:09.818275    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod84608566_23d5_44cd_bf78_b13e83008894.slice/crio-443f090634a8e3ddd6d23a8f0e66d5317e1c2b629eccd928e373ccc921e1e86e WatchSource:0}: Error finding container 443f090634a8e3ddd6d23a8f0e66d5317e1c2b629eccd928e373ccc921e1e86e: Status 404 returned error can't find the container with id 443f090634a8e3ddd6d23a8f0e66d5317e1c2b629eccd928e373ccc921e1e86e
Jul 10 23:28:09.879670 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.879616    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv" event={"ID":"84608566-23d5-44cd-bf78-b13e83008894","Type":"ContainerStarted","Data":"443f090634a8e3ddd6d23a8f0e66d5317e1c2b629eccd928e373ccc921e1e86e"}
Jul 10 23:28:09.881859 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:09.881790    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8" event={"ID":"ba213e27-81b4-4059-b274-a895caac972c","Type":"ContainerStarted","Data":"d46435e71a75718afba1b6650e61e8e7b975a4580833a2af9b59006cca941f1d"}
Jul 10 23:28:10.035623 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:10.035587    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:28:10.035865 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:10.035839    2563 patch_prober.go:28] interesting pod/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 10 23:28:10.035994 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:10.035897    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8" podUID="ba213e27-81b4-4059-b274-a895caac972c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 10 23:28:10.888050 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:10.888006    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv" event={"ID":"84608566-23d5-44cd-bf78-b13e83008894","Type":"ContainerStarted","Data":"3cc355c5ac12192f50110cdff649bf647fd910e16f76b9814340e0a0a65e8fff"}
Jul 10 23:28:10.902049 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:10.901984    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv" podStartSLOduration=1.901965889 podStartE2EDuration="1.901965889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:28:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:28:10.901927704 +0000 UTC m=+1951.788663848" watchObservedRunningTime="2026-07-10 23:28:10.901965889 +0000 UTC m=+1951.788702033"
Jul 10 23:28:11.036430 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:11.036355    2563 patch_prober.go:28] interesting pod/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 10 23:28:11.036617 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:11.036449    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8" podUID="ba213e27-81b4-4059-b274-a895caac972c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 10 23:28:12.035384 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:12.035322    2563 patch_prober.go:28] interesting pod/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 10 23:28:12.035786 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:12.035414    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8" podUID="ba213e27-81b4-4059-b274-a895caac972c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 10 23:28:13.035580 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:13.035538    2563 patch_prober.go:28] interesting pod/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 10 23:28:13.035991 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:13.035598    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8" podUID="ba213e27-81b4-4059-b274-a895caac972c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 10 23:28:14.039423 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:14.039386    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:28:14.039916 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:14.039899    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:28:14.040742 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:14.040723    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:28:18.365140 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:18.365103    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:28:18.468481 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:18.468446    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:28:19.631845 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:19.631809    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"
Jul 10 23:28:19.632280 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:19.632088    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"
Jul 10 23:28:19.644597 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:19.644569    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"
Jul 10 23:28:19.942348 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:19.942262    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"
Jul 10 23:28:26.837995 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:26.837961    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:28:26.940846 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:26.940811    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:28:35.665144 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:35.665048    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:28:36.138601 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:36.138558    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:28:36.239159 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:36.239126    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:28:36.395303 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:36.395214    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:28:41.755635 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:41.755598    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:28:46.738131 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:46.738085    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"]
Jul 10 23:28:46.777380 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:28:46.777336    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:29:01.759747 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:01.759681    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv" podUID="84608566-23d5-44cd-bf78-b13e83008894" containerName="main" containerID="cri-o://3cc355c5ac12192f50110cdff649bf647fd910e16f76b9814340e0a0a65e8fff" gracePeriod=45
Jul 10 23:29:02.024648 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:02.024580    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"
Jul 10 23:29:02.110114 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:02.110077    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84608566-23d5-44cd-bf78-b13e83008894-tls-certs\") pod \"84608566-23d5-44cd-bf78-b13e83008894\" (UID: \"84608566-23d5-44cd-bf78-b13e83008894\") "
Jul 10 23:29:02.110114 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:02.110117    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cr6wd\" (UniqueName: \"kubernetes.io/projected/84608566-23d5-44cd-bf78-b13e83008894-kube-api-access-cr6wd\") pod \"84608566-23d5-44cd-bf78-b13e83008894\" (UID: \"84608566-23d5-44cd-bf78-b13e83008894\") "
Jul 10 23:29:02.110338 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:02.110143    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84608566-23d5-44cd-bf78-b13e83008894-model-cache\") pod \"84608566-23d5-44cd-bf78-b13e83008894\" (UID: \"84608566-23d5-44cd-bf78-b13e83008894\") "
Jul 10 23:29:02.110557 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:02.110523    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84608566-23d5-44cd-bf78-b13e83008894-model-cache" (OuterVolumeSpecName: "model-cache") pod "84608566-23d5-44cd-bf78-b13e83008894" (UID: "84608566-23d5-44cd-bf78-b13e83008894"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:29:02.112637 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:02.112596    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84608566-23d5-44cd-bf78-b13e83008894-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "84608566-23d5-44cd-bf78-b13e83008894" (UID: "84608566-23d5-44cd-bf78-b13e83008894"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:29:02.112806 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:02.112683    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84608566-23d5-44cd-bf78-b13e83008894-kube-api-access-cr6wd" (OuterVolumeSpecName: "kube-api-access-cr6wd") pod "84608566-23d5-44cd-bf78-b13e83008894" (UID: "84608566-23d5-44cd-bf78-b13e83008894"). InnerVolumeSpecName "kube-api-access-cr6wd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:29:02.144489 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:02.144452    2563 generic.go:356] "Generic (PLEG): container finished" podID="84608566-23d5-44cd-bf78-b13e83008894" containerID="3cc355c5ac12192f50110cdff649bf647fd910e16f76b9814340e0a0a65e8fff" exitCode=0
Jul 10 23:29:02.144489 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:02.144495    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"
Jul 10 23:29:02.144741 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:02.144532    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv" event={"ID":"84608566-23d5-44cd-bf78-b13e83008894","Type":"ContainerDied","Data":"3cc355c5ac12192f50110cdff649bf647fd910e16f76b9814340e0a0a65e8fff"}
Jul 10 23:29:02.144741 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:02.144579    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv" event={"ID":"84608566-23d5-44cd-bf78-b13e83008894","Type":"ContainerDied","Data":"443f090634a8e3ddd6d23a8f0e66d5317e1c2b629eccd928e373ccc921e1e86e"}
Jul 10 23:29:02.144741 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:02.144597    2563 scope.go:117] "RemoveContainer" containerID="3cc355c5ac12192f50110cdff649bf647fd910e16f76b9814340e0a0a65e8fff"
Jul 10 23:29:02.153657 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:02.153636    2563 scope.go:117] "RemoveContainer" containerID="3cc355c5ac12192f50110cdff649bf647fd910e16f76b9814340e0a0a65e8fff"
Jul 10 23:29:02.153933 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:29:02.153913    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cc355c5ac12192f50110cdff649bf647fd910e16f76b9814340e0a0a65e8fff\": container with ID starting with 3cc355c5ac12192f50110cdff649bf647fd910e16f76b9814340e0a0a65e8fff not found: ID does not exist" containerID="3cc355c5ac12192f50110cdff649bf647fd910e16f76b9814340e0a0a65e8fff"
Jul 10 23:29:02.153977 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:02.153947    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cc355c5ac12192f50110cdff649bf647fd910e16f76b9814340e0a0a65e8fff"} err="failed to get container status \"3cc355c5ac12192f50110cdff649bf647fd910e16f76b9814340e0a0a65e8fff\": rpc error: code = NotFound desc = could not find container \"3cc355c5ac12192f50110cdff649bf647fd910e16f76b9814340e0a0a65e8fff\": container with ID starting with 3cc355c5ac12192f50110cdff649bf647fd910e16f76b9814340e0a0a65e8fff not found: ID does not exist"
Jul 10 23:29:02.211238 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:02.211191    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84608566-23d5-44cd-bf78-b13e83008894-tmp-dir\") pod \"84608566-23d5-44cd-bf78-b13e83008894\" (UID: \"84608566-23d5-44cd-bf78-b13e83008894\") "
Jul 10 23:29:02.211238 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:02.211244    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84608566-23d5-44cd-bf78-b13e83008894-dshm\") pod \"84608566-23d5-44cd-bf78-b13e83008894\" (UID: \"84608566-23d5-44cd-bf78-b13e83008894\") "
Jul 10 23:29:02.211485 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:02.211314    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84608566-23d5-44cd-bf78-b13e83008894-home\") pod \"84608566-23d5-44cd-bf78-b13e83008894\" (UID: \"84608566-23d5-44cd-bf78-b13e83008894\") "
Jul 10 23:29:02.211485 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:02.211448    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84608566-23d5-44cd-bf78-b13e83008894-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "84608566-23d5-44cd-bf78-b13e83008894" (UID: "84608566-23d5-44cd-bf78-b13e83008894"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:29:02.211632 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:02.211613    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84608566-23d5-44cd-bf78-b13e83008894-tls-certs\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:29:02.211669 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:02.211625    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84608566-23d5-44cd-bf78-b13e83008894-home" (OuterVolumeSpecName: "home") pod "84608566-23d5-44cd-bf78-b13e83008894" (UID: "84608566-23d5-44cd-bf78-b13e83008894"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:29:02.211669 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:02.211639    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cr6wd\" (UniqueName: \"kubernetes.io/projected/84608566-23d5-44cd-bf78-b13e83008894-kube-api-access-cr6wd\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:29:02.211669 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:02.211652    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84608566-23d5-44cd-bf78-b13e83008894-model-cache\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:29:02.211669 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:02.211664    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84608566-23d5-44cd-bf78-b13e83008894-tmp-dir\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:29:02.213643 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:02.213609    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84608566-23d5-44cd-bf78-b13e83008894-dshm" (OuterVolumeSpecName: "dshm") pod "84608566-23d5-44cd-bf78-b13e83008894" (UID: "84608566-23d5-44cd-bf78-b13e83008894"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:29:02.312210 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:02.312113    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84608566-23d5-44cd-bf78-b13e83008894-home\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:29:02.312210 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:02.312144    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84608566-23d5-44cd-bf78-b13e83008894-dshm\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:29:02.464607 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:02.464563    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"]
Jul 10 23:29:02.467976 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:02.467931    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-7e40d510/e2e-lifecycle-model-7e40d510-kserve-65f79b4fff-cfxkv"]
Jul 10 23:29:03.556676 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:03.556637    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84608566-23d5-44cd-bf78-b13e83008894" path="/var/lib/kubelet/pods/84608566-23d5-44cd-bf78-b13e83008894/volumes"
Jul 10 23:29:23.528394 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:23.528334    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:29:23.669487 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:23.669452    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:29:29.223340 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:29.223306    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:29:43.209237 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:43.209199    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:29:43.267785 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:43.267754    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:29:43.470030 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:43.469927    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"]
Jul 10 23:29:43.470201 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:43.470149    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8" podUID="ba213e27-81b4-4059-b274-a895caac972c" containerName="istio-proxy" containerID="cri-o://d46435e71a75718afba1b6650e61e8e7b975a4580833a2af9b59006cca941f1d" gracePeriod=30
Jul 10 23:29:43.485653 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:43.485621    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:29:43.745606 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:43.745567    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:29:44.042619 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:44.042525    2563 patch_prober.go:28] interesting pod/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 10 23:29:44.042619 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:44.042599    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8" podUID="ba213e27-81b4-4059-b274-a895caac972c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 23:29:44.811219 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:44.811182    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:29:44.906883 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:44.906843    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:29:45.006958 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:45.006924    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:29:45.700518 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:45.700484    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:29:45.800317 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:45.800281    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:29:45.909522 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:45.909480    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:29:47.287466 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:47.287431    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:29:48.745980 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:48.745894    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:29:48.840400 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:48.840347    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-credential-socket\") pod \"ba213e27-81b4-4059-b274-a895caac972c\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") "
Jul 10 23:29:48.840587 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:48.840437    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba213e27-81b4-4059-b274-a895caac972c-istio-token\") pod \"ba213e27-81b4-4059-b274-a895caac972c\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") "
Jul 10 23:29:48.840587 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:48.840504    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-istio-envoy\") pod \"ba213e27-81b4-4059-b274-a895caac972c\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") "
Jul 10 23:29:48.840587 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:48.840529    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-workload-certs\") pod \"ba213e27-81b4-4059-b274-a895caac972c\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") "
Jul 10 23:29:48.840587 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:48.840572    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-workload-socket\") pod \"ba213e27-81b4-4059-b274-a895caac972c\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") "
Jul 10 23:29:48.840781 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:48.840603    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7jf6\" (UniqueName: \"kubernetes.io/projected/ba213e27-81b4-4059-b274-a895caac972c-kube-api-access-j7jf6\") pod \"ba213e27-81b4-4059-b274-a895caac972c\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") "
Jul 10 23:29:48.840781 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:48.840644    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba213e27-81b4-4059-b274-a895caac972c-istiod-ca-cert\") pod \"ba213e27-81b4-4059-b274-a895caac972c\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") "
Jul 10 23:29:48.840781 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:48.840646    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ba213e27-81b4-4059-b274-a895caac972c" (UID: "ba213e27-81b4-4059-b274-a895caac972c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:29:48.840781 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:48.840676    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-istio-data\") pod \"ba213e27-81b4-4059-b274-a895caac972c\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") "
Jul 10 23:29:48.840781 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:48.840702    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba213e27-81b4-4059-b274-a895caac972c-istio-podinfo\") pod \"ba213e27-81b4-4059-b274-a895caac972c\" (UID: \"ba213e27-81b4-4059-b274-a895caac972c\") "
Jul 10 23:29:48.841038 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:48.841014    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-credential-socket\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:29:48.841038 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:48.841009    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ba213e27-81b4-4059-b274-a895caac972c" (UID: "ba213e27-81b4-4059-b274-a895caac972c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:29:48.841150 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:48.841093    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba213e27-81b4-4059-b274-a895caac972c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ba213e27-81b4-4059-b274-a895caac972c" (UID: "ba213e27-81b4-4059-b274-a895caac972c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:29:48.841280 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:48.841252    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ba213e27-81b4-4059-b274-a895caac972c" (UID: "ba213e27-81b4-4059-b274-a895caac972c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:29:48.841844 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:48.841823    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-istio-data" (OuterVolumeSpecName: "istio-data") pod "ba213e27-81b4-4059-b274-a895caac972c" (UID: "ba213e27-81b4-4059-b274-a895caac972c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:29:48.843149 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:48.843122    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba213e27-81b4-4059-b274-a895caac972c-kube-api-access-j7jf6" (OuterVolumeSpecName: "kube-api-access-j7jf6") pod "ba213e27-81b4-4059-b274-a895caac972c" (UID: "ba213e27-81b4-4059-b274-a895caac972c"). InnerVolumeSpecName "kube-api-access-j7jf6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:29:48.843344 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:48.843316    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ba213e27-81b4-4059-b274-a895caac972c" (UID: "ba213e27-81b4-4059-b274-a895caac972c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:29:48.843344 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:48.843339    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba213e27-81b4-4059-b274-a895caac972c-istio-token" (OuterVolumeSpecName: "istio-token") pod "ba213e27-81b4-4059-b274-a895caac972c" (UID: "ba213e27-81b4-4059-b274-a895caac972c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:29:48.843669 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:48.843653    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ba213e27-81b4-4059-b274-a895caac972c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ba213e27-81b4-4059-b274-a895caac972c" (UID: "ba213e27-81b4-4059-b274-a895caac972c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 23:29:48.942141 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:48.942096    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ba213e27-81b4-4059-b274-a895caac972c-istio-token\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:29:48.942141 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:48.942129    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-istio-envoy\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:29:48.942141 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:48.942141    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-workload-certs\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:29:48.942141 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:48.942149    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-workload-socket\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:29:48.942141 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:48.942158    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7jf6\" (UniqueName: \"kubernetes.io/projected/ba213e27-81b4-4059-b274-a895caac972c-kube-api-access-j7jf6\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:29:48.942604 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:48.942166    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ba213e27-81b4-4059-b274-a895caac972c-istiod-ca-cert\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:29:48.942604 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:48.942175    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ba213e27-81b4-4059-b274-a895caac972c-istio-data\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:29:48.942604 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:48.942184    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ba213e27-81b4-4059-b274-a895caac972c-istio-podinfo\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:29:49.383299 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:49.383264    2563 generic.go:356] "Generic (PLEG): container finished" podID="ba213e27-81b4-4059-b274-a895caac972c" containerID="d46435e71a75718afba1b6650e61e8e7b975a4580833a2af9b59006cca941f1d" exitCode=0
Jul 10 23:29:49.383543 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:49.383319    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"
Jul 10 23:29:49.383543 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:49.383385    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8" event={"ID":"ba213e27-81b4-4059-b274-a895caac972c","Type":"ContainerDied","Data":"d46435e71a75718afba1b6650e61e8e7b975a4580833a2af9b59006cca941f1d"}
Jul 10 23:29:49.383543 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:49.383443    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8" event={"ID":"ba213e27-81b4-4059-b274-a895caac972c","Type":"ContainerDied","Data":"4be2014b432329634e1e762ca20d76235f76e7e69819b7b6642f0dc209eaca28"}
Jul 10 23:29:49.383543 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:49.383466    2563 scope.go:117] "RemoveContainer" containerID="d46435e71a75718afba1b6650e61e8e7b975a4580833a2af9b59006cca941f1d"
Jul 10 23:29:49.396487 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:49.396401    2563 scope.go:117] "RemoveContainer" containerID="4a4652528b28abf1b458bcb2e701c31cdb26c4a28153a96da93ba3b7f93bf619"
Jul 10 23:29:49.407388 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:49.407337    2563 scope.go:117] "RemoveContainer" containerID="d46435e71a75718afba1b6650e61e8e7b975a4580833a2af9b59006cca941f1d"
Jul 10 23:29:49.407738 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:29:49.407718    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d46435e71a75718afba1b6650e61e8e7b975a4580833a2af9b59006cca941f1d\": container with ID starting with d46435e71a75718afba1b6650e61e8e7b975a4580833a2af9b59006cca941f1d not found: ID does not exist" containerID="d46435e71a75718afba1b6650e61e8e7b975a4580833a2af9b59006cca941f1d"
Jul 10 23:29:49.407831 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:49.407752    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d46435e71a75718afba1b6650e61e8e7b975a4580833a2af9b59006cca941f1d"} err="failed to get container status \"d46435e71a75718afba1b6650e61e8e7b975a4580833a2af9b59006cca941f1d\": rpc error: code = NotFound desc = could not find container \"d46435e71a75718afba1b6650e61e8e7b975a4580833a2af9b59006cca941f1d\": container with ID starting with d46435e71a75718afba1b6650e61e8e7b975a4580833a2af9b59006cca941f1d not found: ID does not exist"
Jul 10 23:29:49.407831 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:49.407777    2563 scope.go:117] "RemoveContainer" containerID="4a4652528b28abf1b458bcb2e701c31cdb26c4a28153a96da93ba3b7f93bf619"
Jul 10 23:29:49.408102 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:29:49.408078    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a4652528b28abf1b458bcb2e701c31cdb26c4a28153a96da93ba3b7f93bf619\": container with ID starting with 4a4652528b28abf1b458bcb2e701c31cdb26c4a28153a96da93ba3b7f93bf619 not found: ID does not exist" containerID="4a4652528b28abf1b458bcb2e701c31cdb26c4a28153a96da93ba3b7f93bf619"
Jul 10 23:29:49.408168 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:49.408103    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a4652528b28abf1b458bcb2e701c31cdb26c4a28153a96da93ba3b7f93bf619"} err="failed to get container status \"4a4652528b28abf1b458bcb2e701c31cdb26c4a28153a96da93ba3b7f93bf619\": rpc error: code = NotFound desc = could not find container \"4a4652528b28abf1b458bcb2e701c31cdb26c4a28153a96da93ba3b7f93bf619\": container with ID starting with 4a4652528b28abf1b458bcb2e701c31cdb26c4a28153a96da93ba3b7f93bf619 not found: ID does not exist"
Jul 10 23:29:49.408285 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:49.408266    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"]
Jul 10 23:29:49.412843 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:49.412809    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7e40d510-openshift-default-774b4f9cb7-6ncs8"]
Jul 10 23:29:49.557998 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:49.557954    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba213e27-81b4-4059-b274-a895caac972c" path="/var/lib/kubelet/pods/ba213e27-81b4-4059-b274-a895caac972c/volumes"
Jul 10 23:29:52.890056 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:52.890016    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:29:52.990650 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:52.990622    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:29:53.594415 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:53.594381    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:29:54.248058 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:54.248027    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:29:54.298412 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:54.298379    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:29:59.176352 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:59.176319    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:29:59.237110 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:59.237071    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:29:59.329956 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:59.329922    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:29:59.429383 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:29:59.429273    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:30:00.137583 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:00.137482    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728770-pt6c8"]
Jul 10 23:30:00.137950 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:00.137934    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84608566-23d5-44cd-bf78-b13e83008894" containerName="main"
Jul 10 23:30:00.138004 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:00.137953    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="84608566-23d5-44cd-bf78-b13e83008894" containerName="main"
Jul 10 23:30:00.138004 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:00.137978    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba213e27-81b4-4059-b274-a895caac972c" containerName="istio-proxy"
Jul 10 23:30:00.138004 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:00.137985    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba213e27-81b4-4059-b274-a895caac972c" containerName="istio-proxy"
Jul 10 23:30:00.138004 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:00.137994    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba213e27-81b4-4059-b274-a895caac972c" containerName="istio-proxy"
Jul 10 23:30:00.138004 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:00.138001    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba213e27-81b4-4059-b274-a895caac972c" containerName="istio-proxy"
Jul 10 23:30:00.138188 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:00.138065    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba213e27-81b4-4059-b274-a895caac972c" containerName="istio-proxy"
Jul 10 23:30:00.138188 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:00.138075    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba213e27-81b4-4059-b274-a895caac972c" containerName="istio-proxy"
Jul 10 23:30:00.138188 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:00.138081    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="84608566-23d5-44cd-bf78-b13e83008894" containerName="main"
Jul 10 23:30:00.142735 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:00.142707    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728770-pt6c8"
Jul 10 23:30:00.145903 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:00.145878    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p89rl\""
Jul 10 23:30:00.146888 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:00.146862    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728770-pt6c8"]
Jul 10 23:30:00.154419 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:00.154397    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728770-pt6c8"
Jul 10 23:30:00.295009 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:00.294984    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728770-pt6c8"]
Jul 10 23:30:00.297930 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:30:00.297893    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda89b8665_87be_45e6_90e9_9ea7e5b9fa95.slice/crio-5f6f11c085b760dd19f14e02aa5442d8e040adf543ce7ad7ea7a30e9fde1e2ab WatchSource:0}: Error finding container 5f6f11c085b760dd19f14e02aa5442d8e040adf543ce7ad7ea7a30e9fde1e2ab: Status 404 returned error can't find the container with id 5f6f11c085b760dd19f14e02aa5442d8e040adf543ce7ad7ea7a30e9fde1e2ab
Jul 10 23:30:00.436858 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:00.436769    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728770-pt6c8" event={"ID":"a89b8665-87be-45e6-90e9-9ea7e5b9fa95","Type":"ContainerStarted","Data":"5f6f11c085b760dd19f14e02aa5442d8e040adf543ce7ad7ea7a30e9fde1e2ab"}
Jul 10 23:30:01.443443 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:01.443401    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728770-pt6c8" event={"ID":"a89b8665-87be-45e6-90e9-9ea7e5b9fa95","Type":"ContainerStarted","Data":"bf28e5b9610d0a6210fd83892b122071de2f9ad62ded867d00dee3a513ad4e0e"}
Jul 10 23:30:01.458406 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:01.458332    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728770-pt6c8" podStartSLOduration=1.45831529 podStartE2EDuration="1.45831529s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:30:01.457764659 +0000 UTC m=+2062.344500846" watchObservedRunningTime="2026-07-10 23:30:01.45831529 +0000 UTC m=+2062.345051432"
Jul 10 23:30:09.898111 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:09.898067    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:30:10.239529 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:10.239493    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:30:31.210490 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:31.210449    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:30:36.726824 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:36.726791    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:30:36.826979 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:36.826940    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:30:37.071054 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.071025    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:30:37.355930 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.355840    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"]
Jul 10 23:30:37.359599 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.359574    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.362323 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.362302    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 10 23:30:37.362513 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.362494    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-43530d-openshift-default-dockercfg-gf2s2\""
Jul 10 23:30:37.370162 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.370126    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"]
Jul 10 23:30:37.372150 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.372127    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-credential-socket\") pod \"e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") " pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.372264 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.372157    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74490ca4-64a9-4b1d-827d-048518febba6-istio-token\") pod \"e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") " pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.372264 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.372181    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-istio-envoy\") pod \"e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") " pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.372264 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.372209    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-workload-socket\") pod \"e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") " pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.372264 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.372247    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74490ca4-64a9-4b1d-827d-048518febba6-istiod-ca-cert\") pod \"e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") " pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.372425 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.372284    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-workload-certs\") pod \"e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") " pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.372425 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.372306    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtlzj\" (UniqueName: \"kubernetes.io/projected/74490ca4-64a9-4b1d-827d-048518febba6-kube-api-access-dtlzj\") pod \"e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") " pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.372425 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.372345    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-istio-data\") pod \"e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") " pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.372425 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.372408    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74490ca4-64a9-4b1d-827d-048518febba6-istio-podinfo\") pod \"e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") " pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.408388 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.408326    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:30:37.473191 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.473158    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-istio-data\") pod \"e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") " pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.473191 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.473196    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74490ca4-64a9-4b1d-827d-048518febba6-istio-podinfo\") pod \"e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") " pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.473603 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.473242    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-credential-socket\") pod \"e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") " pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.473603 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.473262    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74490ca4-64a9-4b1d-827d-048518febba6-istio-token\") pod \"e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") " pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.473603 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.473280    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-istio-envoy\") pod \"e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") " pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.473603 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.473310    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-workload-socket\") pod \"e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") " pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.473603 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.473343    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74490ca4-64a9-4b1d-827d-048518febba6-istiod-ca-cert\") pod \"e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") " pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.473603 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.473419    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-workload-certs\") pod \"e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") " pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.473603 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.473447    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtlzj\" (UniqueName: \"kubernetes.io/projected/74490ca4-64a9-4b1d-827d-048518febba6-kube-api-access-dtlzj\") pod \"e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") " pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.473921 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.473744    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-istio-data\") pod \"e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") " pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.473921 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.473800    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-credential-socket\") pod \"e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") " pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.473921 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.473889    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-workload-socket\") pod \"e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") " pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.474067 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.473983    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-workload-certs\") pod \"e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") " pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.474266 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.474245    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74490ca4-64a9-4b1d-827d-048518febba6-istiod-ca-cert\") pod \"e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") " pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.475877 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.475859    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-istio-envoy\") pod \"e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") " pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.476208 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.476189    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74490ca4-64a9-4b1d-827d-048518febba6-istio-podinfo\") pod \"e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") " pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.484615 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.484578    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74490ca4-64a9-4b1d-827d-048518febba6-istio-token\") pod \"e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") " pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.484837 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.484812    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtlzj\" (UniqueName: \"kubernetes.io/projected/74490ca4-64a9-4b1d-827d-048518febba6-kube-api-access-dtlzj\") pod \"e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") " pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.512943 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.512907    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:30:37.672655 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.672555    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:37.829393 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.829347    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"]
Jul 10 23:30:37.832098 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:30:37.832070    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74490ca4_64a9_4b1d_827d_048518febba6.slice/crio-a38b1bd8e752adac60b181f3afe6a60dce63743be0165febe02257beacd7d2c9 WatchSource:0}: Error finding container a38b1bd8e752adac60b181f3afe6a60dce63743be0165febe02257beacd7d2c9: Status 404 returned error can't find the container with id a38b1bd8e752adac60b181f3afe6a60dce63743be0165febe02257beacd7d2c9
Jul 10 23:30:37.834188 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.834155    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:30:37.834269 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.834222    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:30:37.834269 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:37.834258    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:30:38.623783 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:38.623741    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph" event={"ID":"74490ca4-64a9-4b1d-827d-048518febba6","Type":"ContainerStarted","Data":"6d846d8f925bdc51bc2ddc329c2cb191307660a9c922048955ffe595a3f6d0bf"}
Jul 10 23:30:38.624087 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:38.623837    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph" event={"ID":"74490ca4-64a9-4b1d-827d-048518febba6","Type":"ContainerStarted","Data":"a38b1bd8e752adac60b181f3afe6a60dce63743be0165febe02257beacd7d2c9"}
Jul 10 23:30:38.643309 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:38.643225    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph" podStartSLOduration=1.643209464 podStartE2EDuration="1.643209464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:30:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:30:38.641420292 +0000 UTC m=+2099.528156436" watchObservedRunningTime="2026-07-10 23:30:38.643209464 +0000 UTC m=+2099.529945606"
Jul 10 23:30:38.672767 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:38.672729    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:38.677765 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:38.677736    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:39.629284 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:39.629247    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:39.630496 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:39.630472    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:39.646398 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:39.646352    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/ovn-acl-logging/0.log"
Jul 10 23:30:39.651547 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:39.651522    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/ovn-acl-logging/0.log"
Jul 10 23:30:40.586200 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:40.586162    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:30:40.685007 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:40.684974    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:30:40.755686 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:40.755630    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:30:40.948656 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:40.948565    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:30:41.137523 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:41.137487    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:30:41.238151 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:41.238117    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:30:45.074549 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.074484    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:30:45.410608 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.410513    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:30:45.686226 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.686124    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"]
Jul 10 23:30:45.689844 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.689816    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:45.692308 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.692280    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-56040a-openshift-default-dockercfg-47588\""
Jul 10 23:30:45.701781 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.701755    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"]
Jul 10 23:30:45.711126 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.711095    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:30:45.759787 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.759746    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-istio-envoy\") pod \"e2e-inf-b-56040a-openshift-default-5698975467-9wb2s\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") " pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:45.759976 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.759803    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40b9a981-a0d1-4ec5-9437-495efb299c67-istiod-ca-cert\") pod \"e2e-inf-b-56040a-openshift-default-5698975467-9wb2s\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") " pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:45.759976 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.759837    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-workload-socket\") pod \"e2e-inf-b-56040a-openshift-default-5698975467-9wb2s\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") " pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:45.759976 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.759870    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-istio-data\") pod \"e2e-inf-b-56040a-openshift-default-5698975467-9wb2s\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") " pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:45.759976 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.759940    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-credential-socket\") pod \"e2e-inf-b-56040a-openshift-default-5698975467-9wb2s\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") " pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:45.760183 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.759983    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-workload-certs\") pod \"e2e-inf-b-56040a-openshift-default-5698975467-9wb2s\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") " pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:45.760183 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.760021    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40b9a981-a0d1-4ec5-9437-495efb299c67-istio-podinfo\") pod \"e2e-inf-b-56040a-openshift-default-5698975467-9wb2s\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") " pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:45.760183 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.760115    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp27l\" (UniqueName: \"kubernetes.io/projected/40b9a981-a0d1-4ec5-9437-495efb299c67-kube-api-access-gp27l\") pod \"e2e-inf-b-56040a-openshift-default-5698975467-9wb2s\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") " pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:45.760183 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.760167    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40b9a981-a0d1-4ec5-9437-495efb299c67-istio-token\") pod \"e2e-inf-b-56040a-openshift-default-5698975467-9wb2s\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") " pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:45.860680 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.860645    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40b9a981-a0d1-4ec5-9437-495efb299c67-istio-podinfo\") pod \"e2e-inf-b-56040a-openshift-default-5698975467-9wb2s\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") " pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:45.860897 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.860703    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp27l\" (UniqueName: \"kubernetes.io/projected/40b9a981-a0d1-4ec5-9437-495efb299c67-kube-api-access-gp27l\") pod \"e2e-inf-b-56040a-openshift-default-5698975467-9wb2s\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") " pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:45.860897 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.860726    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40b9a981-a0d1-4ec5-9437-495efb299c67-istio-token\") pod \"e2e-inf-b-56040a-openshift-default-5698975467-9wb2s\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") " pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:45.860897 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.860752    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-istio-envoy\") pod \"e2e-inf-b-56040a-openshift-default-5698975467-9wb2s\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") " pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:45.860897 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.860774    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40b9a981-a0d1-4ec5-9437-495efb299c67-istiod-ca-cert\") pod \"e2e-inf-b-56040a-openshift-default-5698975467-9wb2s\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") " pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:45.861250 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.860890    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-workload-socket\") pod \"e2e-inf-b-56040a-openshift-default-5698975467-9wb2s\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") " pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:45.861329 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.861287    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-istio-data\") pod \"e2e-inf-b-56040a-openshift-default-5698975467-9wb2s\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") " pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:45.861329 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.861227    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-workload-socket\") pod \"e2e-inf-b-56040a-openshift-default-5698975467-9wb2s\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") " pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:45.861466 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.861328    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-credential-socket\") pod \"e2e-inf-b-56040a-openshift-default-5698975467-9wb2s\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") " pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:45.861466 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.861352    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-workload-certs\") pod \"e2e-inf-b-56040a-openshift-default-5698975467-9wb2s\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") " pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:45.861645 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.861625    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-workload-certs\") pod \"e2e-inf-b-56040a-openshift-default-5698975467-9wb2s\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") " pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:45.861722 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.861708    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-credential-socket\") pod \"e2e-inf-b-56040a-openshift-default-5698975467-9wb2s\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") " pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:45.861820 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.861797    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-istio-data\") pod \"e2e-inf-b-56040a-openshift-default-5698975467-9wb2s\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") " pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:45.861871 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.861846    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40b9a981-a0d1-4ec5-9437-495efb299c67-istiod-ca-cert\") pod \"e2e-inf-b-56040a-openshift-default-5698975467-9wb2s\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") " pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:45.863281 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.863252    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-istio-envoy\") pod \"e2e-inf-b-56040a-openshift-default-5698975467-9wb2s\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") " pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:45.863506 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.863489    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40b9a981-a0d1-4ec5-9437-495efb299c67-istio-podinfo\") pod \"e2e-inf-b-56040a-openshift-default-5698975467-9wb2s\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") " pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:45.870897 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.870871    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40b9a981-a0d1-4ec5-9437-495efb299c67-istio-token\") pod \"e2e-inf-b-56040a-openshift-default-5698975467-9wb2s\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") " pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:45.870897 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.870893    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp27l\" (UniqueName: \"kubernetes.io/projected/40b9a981-a0d1-4ec5-9437-495efb299c67-kube-api-access-gp27l\") pod \"e2e-inf-b-56040a-openshift-default-5698975467-9wb2s\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") " pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:45.906026 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:45.905986    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:30:46.001613 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:46.001574    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:46.146451 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:46.146419    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"]
Jul 10 23:30:46.149149 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:30:46.149119    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40b9a981_a0d1_4ec5_9437_495efb299c67.slice/crio-0e7f7f499e79cb0dd695d6088688605fb92e7ced72b452b895f7d9e730db4cdd WatchSource:0}: Error finding container 0e7f7f499e79cb0dd695d6088688605fb92e7ced72b452b895f7d9e730db4cdd: Status 404 returned error can't find the container with id 0e7f7f499e79cb0dd695d6088688605fb92e7ced72b452b895f7d9e730db4cdd
Jul 10 23:30:46.151819 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:46.151770    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:30:46.151905 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:46.151868    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:30:46.151963 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:46.151930    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:30:46.664784 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:46.664731    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s" event={"ID":"40b9a981-a0d1-4ec5-9437-495efb299c67","Type":"ContainerStarted","Data":"9c3ebd9efe0f8b949eda14799a445a9e38d56b0605ff202a9763002fb9596c9d"}
Jul 10 23:30:46.664979 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:46.664798    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s" event={"ID":"40b9a981-a0d1-4ec5-9437-495efb299c67","Type":"ContainerStarted","Data":"0e7f7f499e79cb0dd695d6088688605fb92e7ced72b452b895f7d9e730db4cdd"}
Jul 10 23:30:46.684332 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:46.684273    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s" podStartSLOduration=1.684251111 podStartE2EDuration="1.684251111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:30:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:30:46.682816386 +0000 UTC m=+2107.569552529" watchObservedRunningTime="2026-07-10 23:30:46.684251111 +0000 UTC m=+2107.570987256"
Jul 10 23:30:47.002458 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:47.002424    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:47.006936 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:47.006907    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:47.670086 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:47.670051    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:47.671059 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:47.671040    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:30:48.640287 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:48.640249    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:30:48.719646 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:48.719618    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:30:48.785507 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:48.785478    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:30:49.098296 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:49.098247    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:30:49.199611 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:49.199581    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:30:49.299781 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:49.299722    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:30:53.130673 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:53.130595    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:30:55.671583 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:55.671539    2563 patch_prober.go:28] interesting pod/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 10 23:30:55.672130 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:55.671629    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph" podUID="74490ca4-64a9-4b1d-827d-048518febba6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 10 23:30:57.721726 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:57.721635    2563 generic.go:356] "Generic (PLEG): container finished" podID="74490ca4-64a9-4b1d-827d-048518febba6" containerID="6d846d8f925bdc51bc2ddc329c2cb191307660a9c922048955ffe595a3f6d0bf" exitCode=0
Jul 10 23:30:57.721726 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:57.721701    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph" event={"ID":"74490ca4-64a9-4b1d-827d-048518febba6","Type":"ContainerDied","Data":"6d846d8f925bdc51bc2ddc329c2cb191307660a9c922048955ffe595a3f6d0bf"}
Jul 10 23:30:57.722185 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:57.722112    2563 scope.go:117] "RemoveContainer" containerID="6d846d8f925bdc51bc2ddc329c2cb191307660a9c922048955ffe595a3f6d0bf"
Jul 10 23:30:57.723135 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:57.723112    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:30:57.723191 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:57.723181    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:30:57.723227 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:57.723217    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:30:58.728920 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:58.728886    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph" event={"ID":"74490ca4-64a9-4b1d-827d-048518febba6","Type":"ContainerStarted","Data":"3f56432779c39514276bf94bf08e0f26229c0f38ecb90eb6a688ec623be64812"}
Jul 10 23:30:59.673636 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:59.673588    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:30:59.673872 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:59.673734    2563 patch_prober.go:28] interesting pod/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 10 23:30:59.673872 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:30:59.673781    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph" podUID="74490ca4-64a9-4b1d-827d-048518febba6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 10 23:31:00.009220 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:00.009177    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728770-pt6c8"]
Jul 10 23:31:00.009686 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:00.009400    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728770-pt6c8" podUID="a89b8665-87be-45e6-90e9-9ea7e5b9fa95" containerName="cleanup" containerID="cri-o://bf28e5b9610d0a6210fd83892b122071de2f9ad62ded867d00dee3a513ad4e0e" gracePeriod=30
Jul 10 23:31:00.673483 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:00.673439    2563 patch_prober.go:28] interesting pod/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 10 23:31:00.673798 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:00.673768    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph" podUID="74490ca4-64a9-4b1d-827d-048518febba6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 10 23:31:01.673187 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:01.673143    2563 patch_prober.go:28] interesting pod/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 10 23:31:01.673595 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:01.673218    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph" podUID="74490ca4-64a9-4b1d-827d-048518febba6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 10 23:31:01.863886 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:01.863742    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:31:01.964188 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:01.964034    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:31:02.673702 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:02.673654    2563 patch_prober.go:28] interesting pod/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 10 23:31:02.674166 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:02.673727    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph" podUID="74490ca4-64a9-4b1d-827d-048518febba6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 10 23:31:03.676546 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:03.676511    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:31:03.677031 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:03.677008    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:31:03.677842 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:03.677826    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:31:03.713572 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:03.713527    2563 patch_prober.go:28] interesting pod/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 10 23:31:03.713752 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:03.713594    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s" podUID="40b9a981-a0d1-4ec5-9437-495efb299c67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 10 23:31:04.764030 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:04.763991    2563 generic.go:356] "Generic (PLEG): container finished" podID="40b9a981-a0d1-4ec5-9437-495efb299c67" containerID="9c3ebd9efe0f8b949eda14799a445a9e38d56b0605ff202a9763002fb9596c9d" exitCode=0
Jul 10 23:31:04.764498 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:04.764062    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s" event={"ID":"40b9a981-a0d1-4ec5-9437-495efb299c67","Type":"ContainerDied","Data":"9c3ebd9efe0f8b949eda14799a445a9e38d56b0605ff202a9763002fb9596c9d"}
Jul 10 23:31:04.764594 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:04.764579    2563 scope.go:117] "RemoveContainer" containerID="9c3ebd9efe0f8b949eda14799a445a9e38d56b0605ff202a9763002fb9596c9d"
Jul 10 23:31:04.765541 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:04.765513    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:31:04.765645 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:04.765593    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:31:04.765711 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:04.765643    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:31:05.774570 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:05.774528    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s" event={"ID":"40b9a981-a0d1-4ec5-9437-495efb299c67","Type":"ContainerStarted","Data":"9f442ecc2fa4fc49a3db64e8310b5d2ae98f5aea3ab4145e91ac53722a52c6a7"}
Jul 10 23:31:06.002778 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:06.002736    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:31:06.002992 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:06.002968    2563 patch_prober.go:28] interesting pod/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 10 23:31:06.003050 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:06.003031    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s" podUID="40b9a981-a0d1-4ec5-9437-495efb299c67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 10 23:31:07.002038 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.001997    2563 patch_prober.go:28] interesting pod/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 10 23:31:07.002487 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.002068    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s" podUID="40b9a981-a0d1-4ec5-9437-495efb299c67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 10 23:31:07.516811 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.516757    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"]
Jul 10 23:31:07.521308 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.521284    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"
Jul 10 23:31:07.523824 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.523799    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-43530d\"/\"kube-root-ca.crt\""
Jul 10 23:31:07.525300 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.525052    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-43530d\"/\"openshift-service-ca.crt\""
Jul 10 23:31:07.525300 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.525093    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-43530d\"/\"test-model-43530d-kserve-self-signed-certs\""
Jul 10 23:31:07.525300 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.525052    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-43530d\"/\"default-dockercfg-qslhc\""
Jul 10 23:31:07.527942 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.527918    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"]
Jul 10 23:31:07.543144 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.543114    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:31:07.676562 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.676520    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56097b1c-f9c0-495a-b008-4fba61311fb5-home\") pod \"test-model-43530d-kserve-584c85976c-nprbd\" (UID: \"56097b1c-f9c0-495a-b008-4fba61311fb5\") " pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"
Jul 10 23:31:07.676562 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.676562    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56097b1c-f9c0-495a-b008-4fba61311fb5-model-cache\") pod \"test-model-43530d-kserve-584c85976c-nprbd\" (UID: \"56097b1c-f9c0-495a-b008-4fba61311fb5\") " pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"
Jul 10 23:31:07.676787 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.676593    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56097b1c-f9c0-495a-b008-4fba61311fb5-tmp-dir\") pod \"test-model-43530d-kserve-584c85976c-nprbd\" (UID: \"56097b1c-f9c0-495a-b008-4fba61311fb5\") " pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"
Jul 10 23:31:07.676787 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.676624    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkvrn\" (UniqueName: \"kubernetes.io/projected/56097b1c-f9c0-495a-b008-4fba61311fb5-kube-api-access-gkvrn\") pod \"test-model-43530d-kserve-584c85976c-nprbd\" (UID: \"56097b1c-f9c0-495a-b008-4fba61311fb5\") " pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"
Jul 10 23:31:07.676787 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.676650    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56097b1c-f9c0-495a-b008-4fba61311fb5-tls-certs\") pod \"test-model-43530d-kserve-584c85976c-nprbd\" (UID: \"56097b1c-f9c0-495a-b008-4fba61311fb5\") " pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"
Jul 10 23:31:07.676787 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.676686    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56097b1c-f9c0-495a-b008-4fba61311fb5-dshm\") pod \"test-model-43530d-kserve-584c85976c-nprbd\" (UID: \"56097b1c-f9c0-495a-b008-4fba61311fb5\") " pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"
Jul 10 23:31:07.778098 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.777999    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56097b1c-f9c0-495a-b008-4fba61311fb5-tmp-dir\") pod \"test-model-43530d-kserve-584c85976c-nprbd\" (UID: \"56097b1c-f9c0-495a-b008-4fba61311fb5\") " pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"
Jul 10 23:31:07.778098 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.778053    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkvrn\" (UniqueName: \"kubernetes.io/projected/56097b1c-f9c0-495a-b008-4fba61311fb5-kube-api-access-gkvrn\") pod \"test-model-43530d-kserve-584c85976c-nprbd\" (UID: \"56097b1c-f9c0-495a-b008-4fba61311fb5\") " pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"
Jul 10 23:31:07.778098 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.778098    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56097b1c-f9c0-495a-b008-4fba61311fb5-tls-certs\") pod \"test-model-43530d-kserve-584c85976c-nprbd\" (UID: \"56097b1c-f9c0-495a-b008-4fba61311fb5\") " pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"
Jul 10 23:31:07.778449 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.778137    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56097b1c-f9c0-495a-b008-4fba61311fb5-dshm\") pod \"test-model-43530d-kserve-584c85976c-nprbd\" (UID: \"56097b1c-f9c0-495a-b008-4fba61311fb5\") " pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"
Jul 10 23:31:07.778449 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.778201    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56097b1c-f9c0-495a-b008-4fba61311fb5-home\") pod \"test-model-43530d-kserve-584c85976c-nprbd\" (UID: \"56097b1c-f9c0-495a-b008-4fba61311fb5\") " pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"
Jul 10 23:31:07.778449 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.778227    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56097b1c-f9c0-495a-b008-4fba61311fb5-model-cache\") pod \"test-model-43530d-kserve-584c85976c-nprbd\" (UID: \"56097b1c-f9c0-495a-b008-4fba61311fb5\") " pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"
Jul 10 23:31:07.778609 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.778510    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56097b1c-f9c0-495a-b008-4fba61311fb5-tmp-dir\") pod \"test-model-43530d-kserve-584c85976c-nprbd\" (UID: \"56097b1c-f9c0-495a-b008-4fba61311fb5\") " pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"
Jul 10 23:31:07.778658 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.778618    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56097b1c-f9c0-495a-b008-4fba61311fb5-model-cache\") pod \"test-model-43530d-kserve-584c85976c-nprbd\" (UID: \"56097b1c-f9c0-495a-b008-4fba61311fb5\") " pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"
Jul 10 23:31:07.778658 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.778631    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56097b1c-f9c0-495a-b008-4fba61311fb5-home\") pod \"test-model-43530d-kserve-584c85976c-nprbd\" (UID: \"56097b1c-f9c0-495a-b008-4fba61311fb5\") " pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"
Jul 10 23:31:07.780733 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.780707    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56097b1c-f9c0-495a-b008-4fba61311fb5-dshm\") pod \"test-model-43530d-kserve-584c85976c-nprbd\" (UID: \"56097b1c-f9c0-495a-b008-4fba61311fb5\") " pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"
Jul 10 23:31:07.780910 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.780892    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56097b1c-f9c0-495a-b008-4fba61311fb5-tls-certs\") pod \"test-model-43530d-kserve-584c85976c-nprbd\" (UID: \"56097b1c-f9c0-495a-b008-4fba61311fb5\") " pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"
Jul 10 23:31:07.785725 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.785700    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkvrn\" (UniqueName: \"kubernetes.io/projected/56097b1c-f9c0-495a-b008-4fba61311fb5-kube-api-access-gkvrn\") pod \"test-model-43530d-kserve-584c85976c-nprbd\" (UID: \"56097b1c-f9c0-495a-b008-4fba61311fb5\") " pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"
Jul 10 23:31:07.831942 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:07.831895    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"
Jul 10 23:31:08.002736 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:08.002703    2563 patch_prober.go:28] interesting pod/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 10 23:31:08.003111 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:08.002770    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s" podUID="40b9a981-a0d1-4ec5-9437-495efb299c67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 10 23:31:08.027632 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:08.027604    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"]
Jul 10 23:31:08.029934 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:31:08.029856    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod56097b1c_f9c0_495a_b008_4fba61311fb5.slice/crio-1ea2e18fe453c6b72c3a3d54c64eb4881cfeb52564276979549e3f463494ec70 WatchSource:0}: Error finding container 1ea2e18fe453c6b72c3a3d54c64eb4881cfeb52564276979549e3f463494ec70: Status 404 returned error can't find the container with id 1ea2e18fe453c6b72c3a3d54c64eb4881cfeb52564276979549e3f463494ec70
Jul 10 23:31:08.791736 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:08.791704    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd" event={"ID":"56097b1c-f9c0-495a-b008-4fba61311fb5","Type":"ContainerStarted","Data":"0f1d27b1880eaef906b5bcdce188018da6316fd66b573253795ac6a2648fb932"}
Jul 10 23:31:08.791916 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:08.791749    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd" event={"ID":"56097b1c-f9c0-495a-b008-4fba61311fb5","Type":"ContainerStarted","Data":"1ea2e18fe453c6b72c3a3d54c64eb4881cfeb52564276979549e3f463494ec70"}
Jul 10 23:31:08.807272 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:08.807206    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd" podStartSLOduration=1.807188145 podStartE2EDuration="1.807188145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:31:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:31:08.805765015 +0000 UTC m=+2129.692501159" watchObservedRunningTime="2026-07-10 23:31:08.807188145 +0000 UTC m=+2129.693924288"
Jul 10 23:31:09.002602 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:09.002571    2563 patch_prober.go:28] interesting pod/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 10 23:31:09.002774 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:09.002625    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s" podUID="40b9a981-a0d1-4ec5-9437-495efb299c67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 10 23:31:10.006507 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:10.006463    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:31:10.007003 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:10.006864    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:31:10.007617 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:10.007599    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:31:10.030246 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:10.030211    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:31:10.130546 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:10.130511    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:31:17.832639 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:17.832604    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"
Jul 10 23:31:17.833153 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:17.832654    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"
Jul 10 23:31:17.846450 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:17.846422    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"
Jul 10 23:31:18.854033 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:18.853994    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"
Jul 10 23:31:30.203250 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:30.203220    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728770-pt6c8"
Jul 10 23:31:30.904776 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:30.904740    2563 generic.go:356] "Generic (PLEG): container finished" podID="a89b8665-87be-45e6-90e9-9ea7e5b9fa95" containerID="bf28e5b9610d0a6210fd83892b122071de2f9ad62ded867d00dee3a513ad4e0e" exitCode=137
Jul 10 23:31:30.904983 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:30.904784    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728770-pt6c8"
Jul 10 23:31:30.904983 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:30.904821    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728770-pt6c8" event={"ID":"a89b8665-87be-45e6-90e9-9ea7e5b9fa95","Type":"ContainerDied","Data":"bf28e5b9610d0a6210fd83892b122071de2f9ad62ded867d00dee3a513ad4e0e"}
Jul 10 23:31:30.904983 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:30.904875    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728770-pt6c8" event={"ID":"a89b8665-87be-45e6-90e9-9ea7e5b9fa95","Type":"ContainerDied","Data":"5f6f11c085b760dd19f14e02aa5442d8e040adf543ce7ad7ea7a30e9fde1e2ab"}
Jul 10 23:31:30.904983 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:30.904899    2563 scope.go:117] "RemoveContainer" containerID="bf28e5b9610d0a6210fd83892b122071de2f9ad62ded867d00dee3a513ad4e0e"
Jul 10 23:31:30.913992 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:30.913968    2563 scope.go:117] "RemoveContainer" containerID="bf28e5b9610d0a6210fd83892b122071de2f9ad62ded867d00dee3a513ad4e0e"
Jul 10 23:31:30.914297 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:31:30.914276    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf28e5b9610d0a6210fd83892b122071de2f9ad62ded867d00dee3a513ad4e0e\": container with ID starting with bf28e5b9610d0a6210fd83892b122071de2f9ad62ded867d00dee3a513ad4e0e not found: ID does not exist" containerID="bf28e5b9610d0a6210fd83892b122071de2f9ad62ded867d00dee3a513ad4e0e"
Jul 10 23:31:30.914342 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:30.914307    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf28e5b9610d0a6210fd83892b122071de2f9ad62ded867d00dee3a513ad4e0e"} err="failed to get container status \"bf28e5b9610d0a6210fd83892b122071de2f9ad62ded867d00dee3a513ad4e0e\": rpc error: code = NotFound desc = could not find container \"bf28e5b9610d0a6210fd83892b122071de2f9ad62ded867d00dee3a513ad4e0e\": container with ID starting with bf28e5b9610d0a6210fd83892b122071de2f9ad62ded867d00dee3a513ad4e0e not found: ID does not exist"
Jul 10 23:31:30.923825 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:30.923787    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728770-pt6c8"]
Jul 10 23:31:30.927817 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:30.927786    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728770-pt6c8"]
Jul 10 23:31:31.556528 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:31.556487    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a89b8665-87be-45e6-90e9-9ea7e5b9fa95" path="/var/lib/kubelet/pods/a89b8665-87be-45e6-90e9-9ea7e5b9fa95/volumes"
Jul 10 23:31:46.749445 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:31:46.749410    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:32:12.093072 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:12.093037    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"]
Jul 10 23:32:12.113325 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:12.113291    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:32:13.064557 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:13.064515    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:32:13.333231 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:13.333142    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:32:13.767591 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:13.767542    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:32:13.885145 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:13.885113    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:32:13.985211 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:13.985176    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:32:14.132005 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:14.131873    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:32:18.842295 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:18.842252    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:32:27.114725 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:27.114680    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd" podUID="56097b1c-f9c0-495a-b008-4fba61311fb5" containerName="main" containerID="cri-o://0f1d27b1880eaef906b5bcdce188018da6316fd66b573253795ac6a2648fb932" gracePeriod=45
Jul 10 23:32:27.377871 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:27.377795    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"
Jul 10 23:32:27.528995 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:27.528947    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56097b1c-f9c0-495a-b008-4fba61311fb5-dshm\") pod \"56097b1c-f9c0-495a-b008-4fba61311fb5\" (UID: \"56097b1c-f9c0-495a-b008-4fba61311fb5\") "
Jul 10 23:32:27.528995 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:27.529001    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56097b1c-f9c0-495a-b008-4fba61311fb5-home\") pod \"56097b1c-f9c0-495a-b008-4fba61311fb5\" (UID: \"56097b1c-f9c0-495a-b008-4fba61311fb5\") "
Jul 10 23:32:27.529232 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:27.529109    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkvrn\" (UniqueName: \"kubernetes.io/projected/56097b1c-f9c0-495a-b008-4fba61311fb5-kube-api-access-gkvrn\") pod \"56097b1c-f9c0-495a-b008-4fba61311fb5\" (UID: \"56097b1c-f9c0-495a-b008-4fba61311fb5\") "
Jul 10 23:32:27.529232 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:27.529137    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56097b1c-f9c0-495a-b008-4fba61311fb5-tls-certs\") pod \"56097b1c-f9c0-495a-b008-4fba61311fb5\" (UID: \"56097b1c-f9c0-495a-b008-4fba61311fb5\") "
Jul 10 23:32:27.529232 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:27.529191    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56097b1c-f9c0-495a-b008-4fba61311fb5-model-cache\") pod \"56097b1c-f9c0-495a-b008-4fba61311fb5\" (UID: \"56097b1c-f9c0-495a-b008-4fba61311fb5\") "
Jul 10 23:32:27.529355 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:27.529231    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56097b1c-f9c0-495a-b008-4fba61311fb5-tmp-dir\") pod \"56097b1c-f9c0-495a-b008-4fba61311fb5\" (UID: \"56097b1c-f9c0-495a-b008-4fba61311fb5\") "
Jul 10 23:32:27.529451 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:27.529345    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56097b1c-f9c0-495a-b008-4fba61311fb5-home" (OuterVolumeSpecName: "home") pod "56097b1c-f9c0-495a-b008-4fba61311fb5" (UID: "56097b1c-f9c0-495a-b008-4fba61311fb5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:32:27.529560 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:27.529536    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56097b1c-f9c0-495a-b008-4fba61311fb5-home\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:32:27.529560 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:27.529544    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56097b1c-f9c0-495a-b008-4fba61311fb5-model-cache" (OuterVolumeSpecName: "model-cache") pod "56097b1c-f9c0-495a-b008-4fba61311fb5" (UID: "56097b1c-f9c0-495a-b008-4fba61311fb5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:32:27.529721 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:27.529586    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56097b1c-f9c0-495a-b008-4fba61311fb5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "56097b1c-f9c0-495a-b008-4fba61311fb5" (UID: "56097b1c-f9c0-495a-b008-4fba61311fb5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:32:27.531592 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:27.531564    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56097b1c-f9c0-495a-b008-4fba61311fb5-dshm" (OuterVolumeSpecName: "dshm") pod "56097b1c-f9c0-495a-b008-4fba61311fb5" (UID: "56097b1c-f9c0-495a-b008-4fba61311fb5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:32:27.531710 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:27.531607    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56097b1c-f9c0-495a-b008-4fba61311fb5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "56097b1c-f9c0-495a-b008-4fba61311fb5" (UID: "56097b1c-f9c0-495a-b008-4fba61311fb5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:32:27.531710 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:27.531646    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56097b1c-f9c0-495a-b008-4fba61311fb5-kube-api-access-gkvrn" (OuterVolumeSpecName: "kube-api-access-gkvrn") pod "56097b1c-f9c0-495a-b008-4fba61311fb5" (UID: "56097b1c-f9c0-495a-b008-4fba61311fb5"). InnerVolumeSpecName "kube-api-access-gkvrn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:32:27.630628 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:27.630536    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56097b1c-f9c0-495a-b008-4fba61311fb5-dshm\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:32:27.630628 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:27.630568    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkvrn\" (UniqueName: \"kubernetes.io/projected/56097b1c-f9c0-495a-b008-4fba61311fb5-kube-api-access-gkvrn\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:32:27.630628 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:27.630579    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56097b1c-f9c0-495a-b008-4fba61311fb5-tls-certs\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:32:27.630628 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:27.630589    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56097b1c-f9c0-495a-b008-4fba61311fb5-model-cache\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:32:27.630628 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:27.630599    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56097b1c-f9c0-495a-b008-4fba61311fb5-tmp-dir\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:32:28.202732 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:28.202697    2563 generic.go:356] "Generic (PLEG): container finished" podID="56097b1c-f9c0-495a-b008-4fba61311fb5" containerID="0f1d27b1880eaef906b5bcdce188018da6316fd66b573253795ac6a2648fb932" exitCode=0
Jul 10 23:32:28.203214 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:28.202746    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"
Jul 10 23:32:28.203214 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:28.202791    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd" event={"ID":"56097b1c-f9c0-495a-b008-4fba61311fb5","Type":"ContainerDied","Data":"0f1d27b1880eaef906b5bcdce188018da6316fd66b573253795ac6a2648fb932"}
Jul 10 23:32:28.203214 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:28.202840    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd" event={"ID":"56097b1c-f9c0-495a-b008-4fba61311fb5","Type":"ContainerDied","Data":"1ea2e18fe453c6b72c3a3d54c64eb4881cfeb52564276979549e3f463494ec70"}
Jul 10 23:32:28.203214 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:28.202866    2563 scope.go:117] "RemoveContainer" containerID="0f1d27b1880eaef906b5bcdce188018da6316fd66b573253795ac6a2648fb932"
Jul 10 23:32:28.211714 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:28.211692    2563 scope.go:117] "RemoveContainer" containerID="0f1d27b1880eaef906b5bcdce188018da6316fd66b573253795ac6a2648fb932"
Jul 10 23:32:28.212029 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:32:28.212004    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f1d27b1880eaef906b5bcdce188018da6316fd66b573253795ac6a2648fb932\": container with ID starting with 0f1d27b1880eaef906b5bcdce188018da6316fd66b573253795ac6a2648fb932 not found: ID does not exist" containerID="0f1d27b1880eaef906b5bcdce188018da6316fd66b573253795ac6a2648fb932"
Jul 10 23:32:28.212089 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:28.212038    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f1d27b1880eaef906b5bcdce188018da6316fd66b573253795ac6a2648fb932"} err="failed to get container status \"0f1d27b1880eaef906b5bcdce188018da6316fd66b573253795ac6a2648fb932\": rpc error: code = NotFound desc = could not find container \"0f1d27b1880eaef906b5bcdce188018da6316fd66b573253795ac6a2648fb932\": container with ID starting with 0f1d27b1880eaef906b5bcdce188018da6316fd66b573253795ac6a2648fb932 not found: ID does not exist"
Jul 10 23:32:28.218214 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:28.218184    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"]
Jul 10 23:32:28.221473 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:28.221445    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-43530d/test-model-43530d-kserve-584c85976c-nprbd"]
Jul 10 23:32:29.558018 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:29.557983    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56097b1c-f9c0-495a-b008-4fba61311fb5" path="/var/lib/kubelet/pods/56097b1c-f9c0-495a-b008-4fba61311fb5/volumes"
Jul 10 23:32:53.907086 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:53.907048    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:32:55.716309 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:55.716267    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:32:55.971050 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:55.970950    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"]
Jul 10 23:32:55.971258 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:55.971231    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph" podUID="74490ca4-64a9-4b1d-827d-048518febba6" containerName="istio-proxy" containerID="cri-o://3f56432779c39514276bf94bf08e0f26229c0f38ecb90eb6a688ec623be64812" gracePeriod=30
Jul 10 23:32:55.990083 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:55.990046    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:32:56.247081 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:56.247054    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:32:56.417064 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:56.417028    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:32:56.851527 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:56.851491    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:32:56.954729 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:56.954694    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:32:57.105888 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:32:57.105791    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:01.232346 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.232266    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:33:01.333970 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.333916    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-workload-certs\") pod \"74490ca4-64a9-4b1d-827d-048518febba6\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") "
Jul 10 23:33:01.333970 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.333981    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74490ca4-64a9-4b1d-827d-048518febba6-istiod-ca-cert\") pod \"74490ca4-64a9-4b1d-827d-048518febba6\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") "
Jul 10 23:33:01.334241 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.334017    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74490ca4-64a9-4b1d-827d-048518febba6-istio-token\") pod \"74490ca4-64a9-4b1d-827d-048518febba6\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") "
Jul 10 23:33:01.334241 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.334099    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtlzj\" (UniqueName: \"kubernetes.io/projected/74490ca4-64a9-4b1d-827d-048518febba6-kube-api-access-dtlzj\") pod \"74490ca4-64a9-4b1d-827d-048518febba6\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") "
Jul 10 23:33:01.334241 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.334134    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-istio-data\") pod \"74490ca4-64a9-4b1d-827d-048518febba6\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") "
Jul 10 23:33:01.334241 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.334164    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74490ca4-64a9-4b1d-827d-048518febba6-istio-podinfo\") pod \"74490ca4-64a9-4b1d-827d-048518febba6\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") "
Jul 10 23:33:01.334241 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.334199    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-credential-socket\") pod \"74490ca4-64a9-4b1d-827d-048518febba6\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") "
Jul 10 23:33:01.334544 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.334252    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "74490ca4-64a9-4b1d-827d-048518febba6" (UID: "74490ca4-64a9-4b1d-827d-048518febba6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:33:01.334544 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.334266    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-istio-envoy\") pod \"74490ca4-64a9-4b1d-827d-048518febba6\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") "
Jul 10 23:33:01.334544 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.334389    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-workload-socket\") pod \"74490ca4-64a9-4b1d-827d-048518febba6\" (UID: \"74490ca4-64a9-4b1d-827d-048518febba6\") "
Jul 10 23:33:01.334720 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.334587    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "74490ca4-64a9-4b1d-827d-048518febba6" (UID: "74490ca4-64a9-4b1d-827d-048518febba6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:33:01.334720 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.334520    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74490ca4-64a9-4b1d-827d-048518febba6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "74490ca4-64a9-4b1d-827d-048518febba6" (UID: "74490ca4-64a9-4b1d-827d-048518febba6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:33:01.334842 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.334797    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-credential-socket\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:01.334842 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.334818    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-workload-certs\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:01.334842 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.334834    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74490ca4-64a9-4b1d-827d-048518febba6-istiod-ca-cert\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:01.335180 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.334813    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "74490ca4-64a9-4b1d-827d-048518febba6" (UID: "74490ca4-64a9-4b1d-827d-048518febba6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:33:01.335515 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.335487    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-istio-data" (OuterVolumeSpecName: "istio-data") pod "74490ca4-64a9-4b1d-827d-048518febba6" (UID: "74490ca4-64a9-4b1d-827d-048518febba6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:33:01.336582 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.336555    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/74490ca4-64a9-4b1d-827d-048518febba6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "74490ca4-64a9-4b1d-827d-048518febba6" (UID: "74490ca4-64a9-4b1d-827d-048518febba6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 23:33:01.336704 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.336627    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "74490ca4-64a9-4b1d-827d-048518febba6" (UID: "74490ca4-64a9-4b1d-827d-048518febba6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:33:01.336704 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.336663    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74490ca4-64a9-4b1d-827d-048518febba6-istio-token" (OuterVolumeSpecName: "istio-token") pod "74490ca4-64a9-4b1d-827d-048518febba6" (UID: "74490ca4-64a9-4b1d-827d-048518febba6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:33:01.336704 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.336694    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74490ca4-64a9-4b1d-827d-048518febba6-kube-api-access-dtlzj" (OuterVolumeSpecName: "kube-api-access-dtlzj") pod "74490ca4-64a9-4b1d-827d-048518febba6" (UID: "74490ca4-64a9-4b1d-827d-048518febba6"). InnerVolumeSpecName "kube-api-access-dtlzj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:33:01.372250 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.372207    2563 generic.go:356] "Generic (PLEG): container finished" podID="74490ca4-64a9-4b1d-827d-048518febba6" containerID="3f56432779c39514276bf94bf08e0f26229c0f38ecb90eb6a688ec623be64812" exitCode=0
Jul 10 23:33:01.372475 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.372250    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph" event={"ID":"74490ca4-64a9-4b1d-827d-048518febba6","Type":"ContainerDied","Data":"3f56432779c39514276bf94bf08e0f26229c0f38ecb90eb6a688ec623be64812"}
Jul 10 23:33:01.372475 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.372324    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph" event={"ID":"74490ca4-64a9-4b1d-827d-048518febba6","Type":"ContainerDied","Data":"a38b1bd8e752adac60b181f3afe6a60dce63743be0165febe02257beacd7d2c9"}
Jul 10 23:33:01.372475 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.372265    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"
Jul 10 23:33:01.372475 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.372354    2563 scope.go:117] "RemoveContainer" containerID="3f56432779c39514276bf94bf08e0f26229c0f38ecb90eb6a688ec623be64812"
Jul 10 23:33:01.382484 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.382464    2563 scope.go:117] "RemoveContainer" containerID="6d846d8f925bdc51bc2ddc329c2cb191307660a9c922048955ffe595a3f6d0bf"
Jul 10 23:33:01.391395 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.391354    2563 scope.go:117] "RemoveContainer" containerID="3f56432779c39514276bf94bf08e0f26229c0f38ecb90eb6a688ec623be64812"
Jul 10 23:33:01.391711 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:33:01.391684    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f56432779c39514276bf94bf08e0f26229c0f38ecb90eb6a688ec623be64812\": container with ID starting with 3f56432779c39514276bf94bf08e0f26229c0f38ecb90eb6a688ec623be64812 not found: ID does not exist" containerID="3f56432779c39514276bf94bf08e0f26229c0f38ecb90eb6a688ec623be64812"
Jul 10 23:33:01.391838 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.391721    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f56432779c39514276bf94bf08e0f26229c0f38ecb90eb6a688ec623be64812"} err="failed to get container status \"3f56432779c39514276bf94bf08e0f26229c0f38ecb90eb6a688ec623be64812\": rpc error: code = NotFound desc = could not find container \"3f56432779c39514276bf94bf08e0f26229c0f38ecb90eb6a688ec623be64812\": container with ID starting with 3f56432779c39514276bf94bf08e0f26229c0f38ecb90eb6a688ec623be64812 not found: ID does not exist"
Jul 10 23:33:01.391838 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.391760    2563 scope.go:117] "RemoveContainer" containerID="6d846d8f925bdc51bc2ddc329c2cb191307660a9c922048955ffe595a3f6d0bf"
Jul 10 23:33:01.392300 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:33:01.392226    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d846d8f925bdc51bc2ddc329c2cb191307660a9c922048955ffe595a3f6d0bf\": container with ID starting with 6d846d8f925bdc51bc2ddc329c2cb191307660a9c922048955ffe595a3f6d0bf not found: ID does not exist" containerID="6d846d8f925bdc51bc2ddc329c2cb191307660a9c922048955ffe595a3f6d0bf"
Jul 10 23:33:01.392300 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.392264    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d846d8f925bdc51bc2ddc329c2cb191307660a9c922048955ffe595a3f6d0bf"} err="failed to get container status \"6d846d8f925bdc51bc2ddc329c2cb191307660a9c922048955ffe595a3f6d0bf\": rpc error: code = NotFound desc = could not find container \"6d846d8f925bdc51bc2ddc329c2cb191307660a9c922048955ffe595a3f6d0bf\": container with ID starting with 6d846d8f925bdc51bc2ddc329c2cb191307660a9c922048955ffe595a3f6d0bf not found: ID does not exist"
Jul 10 23:33:01.394629 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.394583    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"]
Jul 10 23:33:01.396779 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.396754    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-43530d-openshift-default-6f5544c66c-bm9ph"]
Jul 10 23:33:01.435913 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.435875    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtlzj\" (UniqueName: \"kubernetes.io/projected/74490ca4-64a9-4b1d-827d-048518febba6-kube-api-access-dtlzj\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:01.435913 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.435909    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-istio-data\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:01.435913 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.435923    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74490ca4-64a9-4b1d-827d-048518febba6-istio-podinfo\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:01.436173 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.435935    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-istio-envoy\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:01.436173 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.435948    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74490ca4-64a9-4b1d-827d-048518febba6-workload-socket\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:01.436173 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.435967    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74490ca4-64a9-4b1d-827d-048518febba6-istio-token\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:01.557384 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:01.557280    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74490ca4-64a9-4b1d-827d-048518febba6" path="/var/lib/kubelet/pods/74490ca4-64a9-4b1d-827d-048518febba6/volumes"
Jul 10 23:33:02.660599 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:02.660563    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:02.754460 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:02.754426    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:02.946043 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:02.945941    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"]
Jul 10 23:33:02.946214 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:02.946159    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s" podUID="40b9a981-a0d1-4ec5-9437-495efb299c67" containerName="istio-proxy" containerID="cri-o://9f442ecc2fa4fc49a3db64e8310b5d2ae98f5aea3ab4145e91ac53722a52c6a7" gracePeriod=30
Jul 10 23:33:02.967625 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:02.967566    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:03.204560 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:03.204460    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:03.549658 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:03.549622    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:03.833247 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:03.833152    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"]
Jul 10 23:33:03.833718 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:03.833551    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a89b8665-87be-45e6-90e9-9ea7e5b9fa95" containerName="cleanup"
Jul 10 23:33:03.833718 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:03.833563    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a89b8665-87be-45e6-90e9-9ea7e5b9fa95" containerName="cleanup"
Jul 10 23:33:03.833718 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:03.833571    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74490ca4-64a9-4b1d-827d-048518febba6" containerName="istio-proxy"
Jul 10 23:33:03.833718 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:03.833576    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="74490ca4-64a9-4b1d-827d-048518febba6" containerName="istio-proxy"
Jul 10 23:33:03.833718 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:03.833582    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56097b1c-f9c0-495a-b008-4fba61311fb5" containerName="main"
Jul 10 23:33:03.833718 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:03.833587    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="56097b1c-f9c0-495a-b008-4fba61311fb5" containerName="main"
Jul 10 23:33:03.833718 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:03.833598    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74490ca4-64a9-4b1d-827d-048518febba6" containerName="istio-proxy"
Jul 10 23:33:03.833718 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:03.833603    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="74490ca4-64a9-4b1d-827d-048518febba6" containerName="istio-proxy"
Jul 10 23:33:03.833718 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:03.833659    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a89b8665-87be-45e6-90e9-9ea7e5b9fa95" containerName="cleanup"
Jul 10 23:33:03.833718 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:03.833671    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="74490ca4-64a9-4b1d-827d-048518febba6" containerName="istio-proxy"
Jul 10 23:33:03.833718 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:03.833677    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="56097b1c-f9c0-495a-b008-4fba61311fb5" containerName="main"
Jul 10 23:33:03.833718 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:03.833683    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="74490ca4-64a9-4b1d-827d-048518febba6" containerName="istio-proxy"
Jul 10 23:33:03.838698 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:03.838674    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:03.841285 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:03.841257    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-7b4cfe-openshift-default-dockercfg-xvkhg\""
Jul 10 23:33:03.849991 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:03.849957    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"]
Jul 10 23:33:03.857225 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:03.857191    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:03.957208 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:03.957163    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec813d98-6cb8-41d1-b947-555d11506c24-istio-podinfo\") pod \"e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") " pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:03.957456 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:03.957237    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec813d98-6cb8-41d1-b947-555d11506c24-istio-token\") pod \"e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") " pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:03.957456 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:03.957268    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-istio-envoy\") pod \"e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") " pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:03.957456 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:03.957291    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-credential-socket\") pod \"e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") " pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:03.957456 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:03.957312    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-workload-certs\") pod \"e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") " pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:03.957456 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:03.957334    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wdjv\" (UniqueName: \"kubernetes.io/projected/ec813d98-6cb8-41d1-b947-555d11506c24-kube-api-access-2wdjv\") pod \"e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") " pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:03.957456 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:03.957443    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-istio-data\") pod \"e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") " pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:03.957739 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:03.957481    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-workload-socket\") pod \"e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") " pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:03.957739 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:03.957552    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec813d98-6cb8-41d1-b947-555d11506c24-istiod-ca-cert\") pod \"e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") " pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:04.058487 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:04.058443    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-workload-socket\") pod \"e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") " pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:04.058685 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:04.058501    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec813d98-6cb8-41d1-b947-555d11506c24-istiod-ca-cert\") pod \"e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") " pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:04.058685 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:04.058540    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec813d98-6cb8-41d1-b947-555d11506c24-istio-podinfo\") pod \"e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") " pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:04.058685 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:04.058582    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec813d98-6cb8-41d1-b947-555d11506c24-istio-token\") pod \"e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") " pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:04.058685 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:04.058598    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-istio-envoy\") pod \"e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") " pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:04.058685 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:04.058614    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-credential-socket\") pod \"e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") " pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:04.058959 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:04.058763    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-workload-certs\") pod \"e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") " pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:04.058959 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:04.058796    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wdjv\" (UniqueName: \"kubernetes.io/projected/ec813d98-6cb8-41d1-b947-555d11506c24-kube-api-access-2wdjv\") pod \"e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") " pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:04.058959 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:04.058828    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-istio-data\") pod \"e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") " pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:04.059120 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:04.058972    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-workload-socket\") pod \"e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") " pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:04.059267 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:04.059177    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-credential-socket\") pod \"e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") " pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:04.059267 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:04.059215    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-workload-certs\") pod \"e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") " pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:04.059267 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:04.059246    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-istio-data\") pod \"e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") " pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:04.059497 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:04.059327    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec813d98-6cb8-41d1-b947-555d11506c24-istiod-ca-cert\") pod \"e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") " pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:04.061208 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:04.061182    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-istio-envoy\") pod \"e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") " pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:04.061695 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:04.061675    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec813d98-6cb8-41d1-b947-555d11506c24-istio-podinfo\") pod \"e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") " pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:04.066415 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:04.066393    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec813d98-6cb8-41d1-b947-555d11506c24-istio-token\") pod \"e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") " pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:04.066705 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:04.066683    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wdjv\" (UniqueName: \"kubernetes.io/projected/ec813d98-6cb8-41d1-b947-555d11506c24-kube-api-access-2wdjv\") pod \"e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") " pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:04.126666 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:04.126577    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:04.151174 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:04.151136    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:04.298323 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:04.298297    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"]
Jul 10 23:33:04.300725 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:33:04.300693    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec813d98_6cb8_41d1_b947_555d11506c24.slice/crio-2fcc27b9329a21fc84a0d4a298e7eaf5fc0290bea2a12ecbd90b6babf59a5ba1 WatchSource:0}: Error finding container 2fcc27b9329a21fc84a0d4a298e7eaf5fc0290bea2a12ecbd90b6babf59a5ba1: Status 404 returned error can't find the container with id 2fcc27b9329a21fc84a0d4a298e7eaf5fc0290bea2a12ecbd90b6babf59a5ba1
Jul 10 23:33:04.303455 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:04.303423    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:33:04.303551 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:04.303489    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:33:04.303551 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:04.303518    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:33:04.391178 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:04.391085    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw" event={"ID":"ec813d98-6cb8-41d1-b947-555d11506c24","Type":"ContainerStarted","Data":"a2ea03f157b18d88caa129804fa9ddcc4392fcb03223be6382c5ab65df8425ea"}
Jul 10 23:33:04.391178 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:04.391152    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw" event={"ID":"ec813d98-6cb8-41d1-b947-555d11506c24","Type":"ContainerStarted","Data":"2fcc27b9329a21fc84a0d4a298e7eaf5fc0290bea2a12ecbd90b6babf59a5ba1"}
Jul 10 23:33:04.409523 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:04.409464    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw" podStartSLOduration=1.409444321 podStartE2EDuration="1.409444321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:33:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:33:04.40793791 +0000 UTC m=+2245.294674053" watchObservedRunningTime="2026-07-10 23:33:04.409444321 +0000 UTC m=+2245.296180466"
Jul 10 23:33:05.151751 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:05.151706    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:05.156659 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:05.156631    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:05.396412 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:05.396354    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:05.397219 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:05.397197    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:06.341642 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:06.341606    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:06.417256 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:06.417226    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:06.474286 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:06.474253    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:06.619581 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:06.619496    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:06.964855 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:06.964760    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:07.065290 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:07.065256    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:08.198162 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.198127    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:33:08.301047 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.301012    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40b9a981-a0d1-4ec5-9437-495efb299c67-istiod-ca-cert\") pod \"40b9a981-a0d1-4ec5-9437-495efb299c67\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") "
Jul 10 23:33:08.301047 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.301064    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-istio-envoy\") pod \"40b9a981-a0d1-4ec5-9437-495efb299c67\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") "
Jul 10 23:33:08.301332 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.301093    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gp27l\" (UniqueName: \"kubernetes.io/projected/40b9a981-a0d1-4ec5-9437-495efb299c67-kube-api-access-gp27l\") pod \"40b9a981-a0d1-4ec5-9437-495efb299c67\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") "
Jul 10 23:33:08.301332 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.301123    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-workload-certs\") pod \"40b9a981-a0d1-4ec5-9437-495efb299c67\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") "
Jul 10 23:33:08.301332 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.301164    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-istio-data\") pod \"40b9a981-a0d1-4ec5-9437-495efb299c67\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") "
Jul 10 23:33:08.301332 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.301192    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40b9a981-a0d1-4ec5-9437-495efb299c67-istio-token\") pod \"40b9a981-a0d1-4ec5-9437-495efb299c67\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") "
Jul 10 23:33:08.301332 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.301234    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-credential-socket\") pod \"40b9a981-a0d1-4ec5-9437-495efb299c67\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") "
Jul 10 23:33:08.301642 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.301346    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-workload-socket\") pod \"40b9a981-a0d1-4ec5-9437-495efb299c67\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") "
Jul 10 23:33:08.301642 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.301412    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40b9a981-a0d1-4ec5-9437-495efb299c67-istio-podinfo\") pod \"40b9a981-a0d1-4ec5-9437-495efb299c67\" (UID: \"40b9a981-a0d1-4ec5-9437-495efb299c67\") "
Jul 10 23:33:08.301642 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.301523    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "40b9a981-a0d1-4ec5-9437-495efb299c67" (UID: "40b9a981-a0d1-4ec5-9437-495efb299c67"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:33:08.301642 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.301564    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40b9a981-a0d1-4ec5-9437-495efb299c67-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "40b9a981-a0d1-4ec5-9437-495efb299c67" (UID: "40b9a981-a0d1-4ec5-9437-495efb299c67"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:33:08.301852 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.301651    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "40b9a981-a0d1-4ec5-9437-495efb299c67" (UID: "40b9a981-a0d1-4ec5-9437-495efb299c67"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:33:08.301852 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.301735    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "40b9a981-a0d1-4ec5-9437-495efb299c67" (UID: "40b9a981-a0d1-4ec5-9437-495efb299c67"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:33:08.301989 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.301969    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-workload-socket\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:08.302209 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.301994    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40b9a981-a0d1-4ec5-9437-495efb299c67-istiod-ca-cert\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:08.302209 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.302008    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-workload-certs\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:08.302209 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.302021    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-credential-socket\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:08.302209 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.302115    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-istio-data" (OuterVolumeSpecName: "istio-data") pod "40b9a981-a0d1-4ec5-9437-495efb299c67" (UID: "40b9a981-a0d1-4ec5-9437-495efb299c67"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:33:08.303636 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.303605    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40b9a981-a0d1-4ec5-9437-495efb299c67-kube-api-access-gp27l" (OuterVolumeSpecName: "kube-api-access-gp27l") pod "40b9a981-a0d1-4ec5-9437-495efb299c67" (UID: "40b9a981-a0d1-4ec5-9437-495efb299c67"). InnerVolumeSpecName "kube-api-access-gp27l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:33:08.303741 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.303680    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40b9a981-a0d1-4ec5-9437-495efb299c67-istio-token" (OuterVolumeSpecName: "istio-token") pod "40b9a981-a0d1-4ec5-9437-495efb299c67" (UID: "40b9a981-a0d1-4ec5-9437-495efb299c67"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:33:08.304112 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.304091    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "40b9a981-a0d1-4ec5-9437-495efb299c67" (UID: "40b9a981-a0d1-4ec5-9437-495efb299c67"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:33:08.304112 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.304101    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/40b9a981-a0d1-4ec5-9437-495efb299c67-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "40b9a981-a0d1-4ec5-9437-495efb299c67" (UID: "40b9a981-a0d1-4ec5-9437-495efb299c67"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 23:33:08.402875 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.402835    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40b9a981-a0d1-4ec5-9437-495efb299c67-istio-podinfo\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:08.402875 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.402868    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-istio-envoy\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:08.402875 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.402878    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gp27l\" (UniqueName: \"kubernetes.io/projected/40b9a981-a0d1-4ec5-9437-495efb299c67-kube-api-access-gp27l\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:08.403129 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.402889    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40b9a981-a0d1-4ec5-9437-495efb299c67-istio-data\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:08.403129 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.402902    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40b9a981-a0d1-4ec5-9437-495efb299c67-istio-token\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:08.414444 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.414403    2563 generic.go:356] "Generic (PLEG): container finished" podID="40b9a981-a0d1-4ec5-9437-495efb299c67" containerID="9f442ecc2fa4fc49a3db64e8310b5d2ae98f5aea3ab4145e91ac53722a52c6a7" exitCode=0
Jul 10 23:33:08.414632 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.414468    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"
Jul 10 23:33:08.414632 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.414481    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s" event={"ID":"40b9a981-a0d1-4ec5-9437-495efb299c67","Type":"ContainerDied","Data":"9f442ecc2fa4fc49a3db64e8310b5d2ae98f5aea3ab4145e91ac53722a52c6a7"}
Jul 10 23:33:08.414632 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.414534    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s" event={"ID":"40b9a981-a0d1-4ec5-9437-495efb299c67","Type":"ContainerDied","Data":"0e7f7f499e79cb0dd695d6088688605fb92e7ced72b452b895f7d9e730db4cdd"}
Jul 10 23:33:08.414632 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.414564    2563 scope.go:117] "RemoveContainer" containerID="9f442ecc2fa4fc49a3db64e8310b5d2ae98f5aea3ab4145e91ac53722a52c6a7"
Jul 10 23:33:08.424446 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.424416    2563 scope.go:117] "RemoveContainer" containerID="9c3ebd9efe0f8b949eda14799a445a9e38d56b0605ff202a9763002fb9596c9d"
Jul 10 23:33:08.433185 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.433166    2563 scope.go:117] "RemoveContainer" containerID="9f442ecc2fa4fc49a3db64e8310b5d2ae98f5aea3ab4145e91ac53722a52c6a7"
Jul 10 23:33:08.433507 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:33:08.433486    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f442ecc2fa4fc49a3db64e8310b5d2ae98f5aea3ab4145e91ac53722a52c6a7\": container with ID starting with 9f442ecc2fa4fc49a3db64e8310b5d2ae98f5aea3ab4145e91ac53722a52c6a7 not found: ID does not exist" containerID="9f442ecc2fa4fc49a3db64e8310b5d2ae98f5aea3ab4145e91ac53722a52c6a7"
Jul 10 23:33:08.433577 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.433519    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f442ecc2fa4fc49a3db64e8310b5d2ae98f5aea3ab4145e91ac53722a52c6a7"} err="failed to get container status \"9f442ecc2fa4fc49a3db64e8310b5d2ae98f5aea3ab4145e91ac53722a52c6a7\": rpc error: code = NotFound desc = could not find container \"9f442ecc2fa4fc49a3db64e8310b5d2ae98f5aea3ab4145e91ac53722a52c6a7\": container with ID starting with 9f442ecc2fa4fc49a3db64e8310b5d2ae98f5aea3ab4145e91ac53722a52c6a7 not found: ID does not exist"
Jul 10 23:33:08.433577 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.433539    2563 scope.go:117] "RemoveContainer" containerID="9c3ebd9efe0f8b949eda14799a445a9e38d56b0605ff202a9763002fb9596c9d"
Jul 10 23:33:08.433809 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:33:08.433791    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c3ebd9efe0f8b949eda14799a445a9e38d56b0605ff202a9763002fb9596c9d\": container with ID starting with 9c3ebd9efe0f8b949eda14799a445a9e38d56b0605ff202a9763002fb9596c9d not found: ID does not exist" containerID="9c3ebd9efe0f8b949eda14799a445a9e38d56b0605ff202a9763002fb9596c9d"
Jul 10 23:33:08.433866 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.433818    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c3ebd9efe0f8b949eda14799a445a9e38d56b0605ff202a9763002fb9596c9d"} err="failed to get container status \"9c3ebd9efe0f8b949eda14799a445a9e38d56b0605ff202a9763002fb9596c9d\": rpc error: code = NotFound desc = could not find container \"9c3ebd9efe0f8b949eda14799a445a9e38d56b0605ff202a9763002fb9596c9d\": container with ID starting with 9c3ebd9efe0f8b949eda14799a445a9e38d56b0605ff202a9763002fb9596c9d not found: ID does not exist"
Jul 10 23:33:08.451582 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.451545    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"]
Jul 10 23:33:08.458739 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:08.458706    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-56040a-openshift-default-5698975467-9wb2s"]
Jul 10 23:33:09.564602 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:09.564561    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40b9a981-a0d1-4ec5-9437-495efb299c67" path="/var/lib/kubelet/pods/40b9a981-a0d1-4ec5-9437-495efb299c67/volumes"
Jul 10 23:33:10.865497 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:10.865457    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:11.098503 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.098462    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:11.331136 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.331107    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:11.607307 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.607215    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"]
Jul 10 23:33:11.607655 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.607640    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40b9a981-a0d1-4ec5-9437-495efb299c67" containerName="istio-proxy"
Jul 10 23:33:11.607721 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.607656    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="40b9a981-a0d1-4ec5-9437-495efb299c67" containerName="istio-proxy"
Jul 10 23:33:11.607721 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.607670    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40b9a981-a0d1-4ec5-9437-495efb299c67" containerName="istio-proxy"
Jul 10 23:33:11.607721 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.607675    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="40b9a981-a0d1-4ec5-9437-495efb299c67" containerName="istio-proxy"
Jul 10 23:33:11.607875 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.607758    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="40b9a981-a0d1-4ec5-9437-495efb299c67" containerName="istio-proxy"
Jul 10 23:33:11.607875 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.607768    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="40b9a981-a0d1-4ec5-9437-495efb299c67" containerName="istio-proxy"
Jul 10 23:33:11.612558 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.612533    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:11.614939 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.614920    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-38233f-openshift-default-dockercfg-k5bd9\""
Jul 10 23:33:11.622891 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.622860    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"]
Jul 10 23:33:11.633460 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.633419    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:11.733487 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.733451    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-workload-socket\") pod \"e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") " pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:11.733487 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.733490    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-istio-data\") pod \"e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") " pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:11.733707 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.733519    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-credential-socket\") pod \"e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") " pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:11.733707 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.733602    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e76fa989-900e-47e1-9470-678dc7c46d4d-istio-token\") pod \"e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") " pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:11.733707 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.733682    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7dz9\" (UniqueName: \"kubernetes.io/projected/e76fa989-900e-47e1-9470-678dc7c46d4d-kube-api-access-w7dz9\") pod \"e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") " pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:11.733836 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.733706    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-istio-envoy\") pod \"e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") " pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:11.733836 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.733726    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-workload-certs\") pod \"e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") " pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:11.733836 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.733741    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e76fa989-900e-47e1-9470-678dc7c46d4d-istio-podinfo\") pod \"e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") " pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:11.733836 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.733775    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e76fa989-900e-47e1-9470-678dc7c46d4d-istiod-ca-cert\") pod \"e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") " pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:11.768209 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.768166    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:11.834533 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.834497    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-credential-socket\") pod \"e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") " pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:11.834731 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.834548    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e76fa989-900e-47e1-9470-678dc7c46d4d-istio-token\") pod \"e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") " pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:11.834731 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.834605    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7dz9\" (UniqueName: \"kubernetes.io/projected/e76fa989-900e-47e1-9470-678dc7c46d4d-kube-api-access-w7dz9\") pod \"e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") " pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:11.834731 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.834632    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-istio-envoy\") pod \"e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") " pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:11.834731 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.834656    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-workload-certs\") pod \"e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") " pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:11.834731 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.834678    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e76fa989-900e-47e1-9470-678dc7c46d4d-istio-podinfo\") pod \"e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") " pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:11.834731 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.834698    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e76fa989-900e-47e1-9470-678dc7c46d4d-istiod-ca-cert\") pod \"e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") " pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:11.835039 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.834761    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-workload-socket\") pod \"e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") " pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:11.835039 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.834792    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-istio-data\") pod \"e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") " pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:11.835039 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.834985    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-credential-socket\") pod \"e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") " pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:11.835199 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.835104    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-workload-certs\") pod \"e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") " pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:11.835199 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.835179    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-istio-data\") pod \"e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") " pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:11.835293 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.835246    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-workload-socket\") pod \"e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") " pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:11.835484 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.835464    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e76fa989-900e-47e1-9470-678dc7c46d4d-istiod-ca-cert\") pod \"e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") " pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:11.837269 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.837248    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-istio-envoy\") pod \"e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") " pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:11.837418 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.837400    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e76fa989-900e-47e1-9470-678dc7c46d4d-istio-podinfo\") pod \"e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") " pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:11.842826 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.842796    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e76fa989-900e-47e1-9470-678dc7c46d4d-istio-token\") pod \"e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") " pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:11.842949 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.842796    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7dz9\" (UniqueName: \"kubernetes.io/projected/e76fa989-900e-47e1-9470-678dc7c46d4d-kube-api-access-w7dz9\") pod \"e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") " pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:11.924139 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:11.924032    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:12.068299 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:12.068268    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"]
Jul 10 23:33:12.070987 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:33:12.070954    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode76fa989_900e_47e1_9470_678dc7c46d4d.slice/crio-6d5205c34a25cf93051162e056a9ef5c1bf0bda0168eeeebab7b87ffc9a67ed2 WatchSource:0}: Error finding container 6d5205c34a25cf93051162e056a9ef5c1bf0bda0168eeeebab7b87ffc9a67ed2: Status 404 returned error can't find the container with id 6d5205c34a25cf93051162e056a9ef5c1bf0bda0168eeeebab7b87ffc9a67ed2
Jul 10 23:33:12.073450 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:12.073406    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:33:12.073546 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:12.073506    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:33:12.073584 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:12.073547    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:33:12.437180 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:12.437147    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5" event={"ID":"e76fa989-900e-47e1-9470-678dc7c46d4d","Type":"ContainerStarted","Data":"67e73a4754a4e7a8238e8e16d9923deea7ceb6901ebbbcda36440324308b76d2"}
Jul 10 23:33:12.437387 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:12.437202    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5" event={"ID":"e76fa989-900e-47e1-9470-678dc7c46d4d","Type":"ContainerStarted","Data":"6d5205c34a25cf93051162e056a9ef5c1bf0bda0168eeeebab7b87ffc9a67ed2"}
Jul 10 23:33:12.924294 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:12.924251    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:13.930536 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:13.930505    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:13.949505 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:13.949449    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5" podStartSLOduration=2.949432754 podStartE2EDuration="2.949432754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:33:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:33:12.454476044 +0000 UTC m=+2253.341212188" watchObservedRunningTime="2026-07-10 23:33:13.949432754 +0000 UTC m=+2254.836168897"
Jul 10 23:33:14.379611 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:14.379573    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:14.449236 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:14.449204    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:14.450588 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:14.450561    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:14.453789 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:14.453757    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-38233f-6fc847547c-f5vmp"]
Jul 10 23:33:14.457209 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:14.457179    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:14.470715 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:14.470684    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-38233f-6fc847547c-f5vmp"
Jul 10 23:33:14.477600 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:14.476188    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fxg5k\""
Jul 10 23:33:14.477600 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:14.476222    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 10 23:33:14.477600 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:14.476519    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-38233f-serving-cert\""
Jul 10 23:33:14.477600 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:14.476724    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 23:33:14.477600 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:14.476950    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-38233f-6fc847547c-f5vmp"]
Jul 10 23:33:14.521124 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:14.521069    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:14.562208 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:14.562172    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1877470f-95e9-48d2-a963-787e2f3bd2e1-openshift-service-ca\") pod \"maas-api-e2e-api-b-38233f-6fc847547c-f5vmp\" (UID: \"1877470f-95e9-48d2-a963-787e2f3bd2e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-38233f-6fc847547c-f5vmp"
Jul 10 23:33:14.562441 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:14.562235    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l68dc\" (UniqueName: \"kubernetes.io/projected/1877470f-95e9-48d2-a963-787e2f3bd2e1-kube-api-access-l68dc\") pod \"maas-api-e2e-api-b-38233f-6fc847547c-f5vmp\" (UID: \"1877470f-95e9-48d2-a963-787e2f3bd2e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-38233f-6fc847547c-f5vmp"
Jul 10 23:33:14.562441 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:14.562322    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1877470f-95e9-48d2-a963-787e2f3bd2e1-maas-api-tls\") pod \"maas-api-e2e-api-b-38233f-6fc847547c-f5vmp\" (UID: \"1877470f-95e9-48d2-a963-787e2f3bd2e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-38233f-6fc847547c-f5vmp"
Jul 10 23:33:14.634264 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:14.634163    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:14.663295 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:14.663254    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1877470f-95e9-48d2-a963-787e2f3bd2e1-maas-api-tls\") pod \"maas-api-e2e-api-b-38233f-6fc847547c-f5vmp\" (UID: \"1877470f-95e9-48d2-a963-787e2f3bd2e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-38233f-6fc847547c-f5vmp"
Jul 10 23:33:14.663481 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:14.663339    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1877470f-95e9-48d2-a963-787e2f3bd2e1-openshift-service-ca\") pod \"maas-api-e2e-api-b-38233f-6fc847547c-f5vmp\" (UID: \"1877470f-95e9-48d2-a963-787e2f3bd2e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-38233f-6fc847547c-f5vmp"
Jul 10 23:33:14.663481 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:14.663405    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l68dc\" (UniqueName: \"kubernetes.io/projected/1877470f-95e9-48d2-a963-787e2f3bd2e1-kube-api-access-l68dc\") pod \"maas-api-e2e-api-b-38233f-6fc847547c-f5vmp\" (UID: \"1877470f-95e9-48d2-a963-787e2f3bd2e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-38233f-6fc847547c-f5vmp"
Jul 10 23:33:14.664108 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:14.664086    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1877470f-95e9-48d2-a963-787e2f3bd2e1-openshift-service-ca\") pod \"maas-api-e2e-api-b-38233f-6fc847547c-f5vmp\" (UID: \"1877470f-95e9-48d2-a963-787e2f3bd2e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-38233f-6fc847547c-f5vmp"
Jul 10 23:33:14.666035 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:14.666013    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1877470f-95e9-48d2-a963-787e2f3bd2e1-maas-api-tls\") pod \"maas-api-e2e-api-b-38233f-6fc847547c-f5vmp\" (UID: \"1877470f-95e9-48d2-a963-787e2f3bd2e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-38233f-6fc847547c-f5vmp"
Jul 10 23:33:14.670643 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:14.670616    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l68dc\" (UniqueName: \"kubernetes.io/projected/1877470f-95e9-48d2-a963-787e2f3bd2e1-kube-api-access-l68dc\") pod \"maas-api-e2e-api-b-38233f-6fc847547c-f5vmp\" (UID: \"1877470f-95e9-48d2-a963-787e2f3bd2e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-38233f-6fc847547c-f5vmp"
Jul 10 23:33:14.791424 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:14.791383    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-38233f-6fc847547c-f5vmp"
Jul 10 23:33:14.939405 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:14.939379    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-38233f-6fc847547c-f5vmp"]
Jul 10 23:33:14.941698 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:33:14.941671    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1877470f_95e9_48d2_a963_787e2f3bd2e1.slice/crio-1ac572b2b533b222d21280472dd61fb97907c982ba76a05efe6f3c20725c83f3 WatchSource:0}: Error finding container 1ac572b2b533b222d21280472dd61fb97907c982ba76a05efe6f3c20725c83f3: Status 404 returned error can't find the container with id 1ac572b2b533b222d21280472dd61fb97907c982ba76a05efe6f3c20725c83f3
Jul 10 23:33:14.943259 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:14.943238    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 23:33:14.949243 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:14.949221    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:15.014633 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:15.014594    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:15.457325 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:15.457289    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-38233f-6fc847547c-f5vmp" event={"ID":"1877470f-95e9-48d2-a963-787e2f3bd2e1","Type":"ContainerStarted","Data":"c0f84d7c32a705412acaa636c264751866da94ee7fc85273e47e14c031376c6b"}
Jul 10 23:33:15.457544 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:15.457356    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-38233f-6fc847547c-f5vmp" event={"ID":"1877470f-95e9-48d2-a963-787e2f3bd2e1","Type":"ContainerStarted","Data":"1ac572b2b533b222d21280472dd61fb97907c982ba76a05efe6f3c20725c83f3"}
Jul 10 23:33:15.457629 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:15.457609    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-38233f-6fc847547c-f5vmp"
Jul 10 23:33:15.475039 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:15.474969    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-38233f-6fc847547c-f5vmp" podStartSLOduration=1.150930449 podStartE2EDuration="1.474944552s" totalImagesPullingTime="324.014103ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:33:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:33:15.471747241 +0000 UTC m=+2256.358483386" watchObservedRunningTime="2026-07-10 23:33:15.474944552 +0000 UTC m=+2256.361680696"
Jul 10 23:33:18.903578 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:18.903545    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:19.086269 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:19.086232    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:19.186640 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:19.186516    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:21.473883 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:21.473849    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-38233f-6fc847547c-f5vmp"
Jul 10 23:33:23.508997 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:23.508962    2563 generic.go:356] "Generic (PLEG): container finished" podID="ec813d98-6cb8-41d1-b947-555d11506c24" containerID="a2ea03f157b18d88caa129804fa9ddcc4392fcb03223be6382c5ab65df8425ea" exitCode=0
Jul 10 23:33:23.509446 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:23.509030    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw" event={"ID":"ec813d98-6cb8-41d1-b947-555d11506c24","Type":"ContainerDied","Data":"a2ea03f157b18d88caa129804fa9ddcc4392fcb03223be6382c5ab65df8425ea"}
Jul 10 23:33:23.509508 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:23.509492    2563 scope.go:117] "RemoveContainer" containerID="a2ea03f157b18d88caa129804fa9ddcc4392fcb03223be6382c5ab65df8425ea"
Jul 10 23:33:23.510593 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:23.510565    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:33:23.510669 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:23.510636    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:33:23.510705 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:23.510668    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:33:24.515665 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:24.515629    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw" event={"ID":"ec813d98-6cb8-41d1-b947-555d11506c24","Type":"ContainerStarted","Data":"793b6844efc03dd1d03113e00361ce37860ab087abfa4ca067f3cfa2633792e7"}
Jul 10 23:33:25.152119 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:25.152073    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:25.152553 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:25.152520    2563 patch_prober.go:28] interesting pod/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 10 23:33:25.152679 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:25.152593    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw" podUID="ec813d98-6cb8-41d1-b947-555d11506c24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 10 23:33:26.153389 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:26.152179    2563 patch_prober.go:28] interesting pod/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 10 23:33:26.153389 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:26.152251    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw" podUID="ec813d98-6cb8-41d1-b947-555d11506c24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 10 23:33:27.109541 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:27.109499    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:27.155815 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:27.155776    2563 patch_prober.go:28] interesting pod/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 10 23:33:27.156454 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:27.156418    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw" podUID="ec813d98-6cb8-41d1-b947-555d11506c24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 10 23:33:27.517563 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:27.517524    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:27.649397 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:27.649323    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:27.750165 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:27.750111    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:27.895557 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:27.895438    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:28.151877 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:28.151766    2563 patch_prober.go:28] interesting pod/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 10 23:33:28.151877 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:28.151854    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw" podUID="ec813d98-6cb8-41d1-b947-555d11506c24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 10 23:33:29.156550 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:29.156515    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:29.157015 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:29.156920    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:29.157755 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:29.157737    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:30.490586 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:30.490536    2563 patch_prober.go:28] interesting pod/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 10 23:33:30.491111 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:30.490596    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5" podUID="e76fa989-900e-47e1-9470-678dc7c46d4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 10 23:33:32.565001 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:32.564963    2563 generic.go:356] "Generic (PLEG): container finished" podID="e76fa989-900e-47e1-9470-678dc7c46d4d" containerID="67e73a4754a4e7a8238e8e16d9923deea7ceb6901ebbbcda36440324308b76d2" exitCode=0
Jul 10 23:33:32.565571 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:32.565016    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5" event={"ID":"e76fa989-900e-47e1-9470-678dc7c46d4d","Type":"ContainerDied","Data":"67e73a4754a4e7a8238e8e16d9923deea7ceb6901ebbbcda36440324308b76d2"}
Jul 10 23:33:32.565571 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:32.565544    2563 scope.go:117] "RemoveContainer" containerID="67e73a4754a4e7a8238e8e16d9923deea7ceb6901ebbbcda36440324308b76d2"
Jul 10 23:33:32.566610 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:32.566584    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:33:32.566681 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:32.566661    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:33:32.566718 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:32.566692    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:33:33.217142 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:33.217103    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:33.317675 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:33.317646    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:33.482385 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:33.481940    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"]
Jul 10 23:33:33.482385 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:33.482218    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw" podUID="ec813d98-6cb8-41d1-b947-555d11506c24" containerName="istio-proxy" containerID="cri-o://793b6844efc03dd1d03113e00361ce37860ab087abfa4ca067f3cfa2633792e7" gracePeriod=30
Jul 10 23:33:33.489261 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:33.489213    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:33.600406 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:33.600341    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5" event={"ID":"e76fa989-900e-47e1-9470-678dc7c46d4d","Type":"ContainerStarted","Data":"0fa6cb8ff63aa0ba6f04c5b877b81b678e8b3ab40475dd5d96bad5cd88d5d5b0"}
Jul 10 23:33:33.713993 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:33.713957    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:33.885839 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:33.885719    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:33.924826 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:33.924786    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:33.925037 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:33.925021    2563 patch_prober.go:28] interesting pod/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 10 23:33:33.925103 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:33.925069    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5" podUID="e76fa989-900e-47e1-9470-678dc7c46d4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 10 23:33:34.306718 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:34.306689    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:34.406705 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:34.406671    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:34.508187 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:34.508147    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:34.642132 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:34.642030    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-38233f-6fc847547c-f5vmp"]
Jul 10 23:33:34.642607 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:34.642258    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-38233f-6fc847547c-f5vmp" podUID="1877470f-95e9-48d2-a963-787e2f3bd2e1" containerName="maas-api" containerID="cri-o://c0f84d7c32a705412acaa636c264751866da94ee7fc85273e47e14c031376c6b" gracePeriod=30
Jul 10 23:33:34.664808 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:34.663770    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:34.925008 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:34.924972    2563 patch_prober.go:28] interesting pod/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 10 23:33:34.925165 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:34.925029    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5" podUID="e76fa989-900e-47e1-9470-678dc7c46d4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 10 23:33:34.929378 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:34.929335    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-38233f-6fc847547c-f5vmp"
Jul 10 23:33:34.984732 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:34.984696    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1877470f-95e9-48d2-a963-787e2f3bd2e1-openshift-service-ca\") pod \"1877470f-95e9-48d2-a963-787e2f3bd2e1\" (UID: \"1877470f-95e9-48d2-a963-787e2f3bd2e1\") "
Jul 10 23:33:34.984912 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:34.984759    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l68dc\" (UniqueName: \"kubernetes.io/projected/1877470f-95e9-48d2-a963-787e2f3bd2e1-kube-api-access-l68dc\") pod \"1877470f-95e9-48d2-a963-787e2f3bd2e1\" (UID: \"1877470f-95e9-48d2-a963-787e2f3bd2e1\") "
Jul 10 23:33:34.984912 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:34.984789    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1877470f-95e9-48d2-a963-787e2f3bd2e1-maas-api-tls\") pod \"1877470f-95e9-48d2-a963-787e2f3bd2e1\" (UID: \"1877470f-95e9-48d2-a963-787e2f3bd2e1\") "
Jul 10 23:33:34.985114 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:34.985087    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1877470f-95e9-48d2-a963-787e2f3bd2e1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1877470f-95e9-48d2-a963-787e2f3bd2e1" (UID: "1877470f-95e9-48d2-a963-787e2f3bd2e1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:33:34.987198 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:34.987166    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1877470f-95e9-48d2-a963-787e2f3bd2e1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1877470f-95e9-48d2-a963-787e2f3bd2e1" (UID: "1877470f-95e9-48d2-a963-787e2f3bd2e1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:33:34.987198 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:34.987183    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1877470f-95e9-48d2-a963-787e2f3bd2e1-kube-api-access-l68dc" (OuterVolumeSpecName: "kube-api-access-l68dc") pod "1877470f-95e9-48d2-a963-787e2f3bd2e1" (UID: "1877470f-95e9-48d2-a963-787e2f3bd2e1"). InnerVolumeSpecName "kube-api-access-l68dc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:33:35.085773 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:35.085733    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1877470f-95e9-48d2-a963-787e2f3bd2e1-openshift-service-ca\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:35.085773 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:35.085770    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l68dc\" (UniqueName: \"kubernetes.io/projected/1877470f-95e9-48d2-a963-787e2f3bd2e1-kube-api-access-l68dc\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:35.085773 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:35.085782    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1877470f-95e9-48d2-a963-787e2f3bd2e1-maas-api-tls\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:35.611737 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:35.611695    2563 generic.go:356] "Generic (PLEG): container finished" podID="1877470f-95e9-48d2-a963-787e2f3bd2e1" containerID="c0f84d7c32a705412acaa636c264751866da94ee7fc85273e47e14c031376c6b" exitCode=0
Jul 10 23:33:35.611737 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:35.611739    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-38233f-6fc847547c-f5vmp" event={"ID":"1877470f-95e9-48d2-a963-787e2f3bd2e1","Type":"ContainerDied","Data":"c0f84d7c32a705412acaa636c264751866da94ee7fc85273e47e14c031376c6b"}
Jul 10 23:33:35.611998 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:35.611746    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-38233f-6fc847547c-f5vmp"
Jul 10 23:33:35.611998 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:35.611785    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-38233f-6fc847547c-f5vmp" event={"ID":"1877470f-95e9-48d2-a963-787e2f3bd2e1","Type":"ContainerDied","Data":"1ac572b2b533b222d21280472dd61fb97907c982ba76a05efe6f3c20725c83f3"}
Jul 10 23:33:35.611998 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:35.611805    2563 scope.go:117] "RemoveContainer" containerID="c0f84d7c32a705412acaa636c264751866da94ee7fc85273e47e14c031376c6b"
Jul 10 23:33:35.621054 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:35.621033    2563 scope.go:117] "RemoveContainer" containerID="c0f84d7c32a705412acaa636c264751866da94ee7fc85273e47e14c031376c6b"
Jul 10 23:33:35.621409 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:33:35.621389    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0f84d7c32a705412acaa636c264751866da94ee7fc85273e47e14c031376c6b\": container with ID starting with c0f84d7c32a705412acaa636c264751866da94ee7fc85273e47e14c031376c6b not found: ID does not exist" containerID="c0f84d7c32a705412acaa636c264751866da94ee7fc85273e47e14c031376c6b"
Jul 10 23:33:35.621482 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:35.621420    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0f84d7c32a705412acaa636c264751866da94ee7fc85273e47e14c031376c6b"} err="failed to get container status \"c0f84d7c32a705412acaa636c264751866da94ee7fc85273e47e14c031376c6b\": rpc error: code = NotFound desc = could not find container \"c0f84d7c32a705412acaa636c264751866da94ee7fc85273e47e14c031376c6b\": container with ID starting with c0f84d7c32a705412acaa636c264751866da94ee7fc85273e47e14c031376c6b not found: ID does not exist"
Jul 10 23:33:35.628491 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:35.628454    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-38233f-6fc847547c-f5vmp"]
Jul 10 23:33:35.631098 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:35.631067    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-38233f-6fc847547c-f5vmp"]
Jul 10 23:33:35.924834 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:35.924741    2563 patch_prober.go:28] interesting pod/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 10 23:33:35.924834 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:35.924793    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5" podUID="e76fa989-900e-47e1-9470-678dc7c46d4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 10 23:33:36.925203 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:36.925164    2563 patch_prober.go:28] interesting pod/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 10 23:33:36.925635 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:36.925223    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5" podUID="e76fa989-900e-47e1-9470-678dc7c46d4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 10 23:33:37.558078 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:37.558043    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1877470f-95e9-48d2-a963-787e2f3bd2e1" path="/var/lib/kubelet/pods/1877470f-95e9-48d2-a963-787e2f3bd2e1/volumes"
Jul 10 23:33:37.928424 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:37.928338    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:37.928853 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:37.928830    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:37.929546 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:37.929526    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:38.629385 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.629324    2563 generic.go:356] "Generic (PLEG): container finished" podID="ec813d98-6cb8-41d1-b947-555d11506c24" containerID="793b6844efc03dd1d03113e00361ce37860ab087abfa4ca067f3cfa2633792e7" exitCode=0
Jul 10 23:33:38.629565 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.629399    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw" event={"ID":"ec813d98-6cb8-41d1-b947-555d11506c24","Type":"ContainerDied","Data":"793b6844efc03dd1d03113e00361ce37860ab087abfa4ca067f3cfa2633792e7"}
Jul 10 23:33:38.629565 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.629461    2563 scope.go:117] "RemoveContainer" containerID="a2ea03f157b18d88caa129804fa9ddcc4392fcb03223be6382c5ab65df8425ea"
Jul 10 23:33:38.761590 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.761513    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:38.824670 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.824627    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec813d98-6cb8-41d1-b947-555d11506c24-istiod-ca-cert\") pod \"ec813d98-6cb8-41d1-b947-555d11506c24\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") "
Jul 10 23:33:38.824861 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.824683    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-workload-socket\") pod \"ec813d98-6cb8-41d1-b947-555d11506c24\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") "
Jul 10 23:33:38.824861 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.824719    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-istio-data\") pod \"ec813d98-6cb8-41d1-b947-555d11506c24\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") "
Jul 10 23:33:38.824861 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.824780    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec813d98-6cb8-41d1-b947-555d11506c24-istio-podinfo\") pod \"ec813d98-6cb8-41d1-b947-555d11506c24\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") "
Jul 10 23:33:38.824861 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.824805    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wdjv\" (UniqueName: \"kubernetes.io/projected/ec813d98-6cb8-41d1-b947-555d11506c24-kube-api-access-2wdjv\") pod \"ec813d98-6cb8-41d1-b947-555d11506c24\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") "
Jul 10 23:33:38.825094 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.824901    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec813d98-6cb8-41d1-b947-555d11506c24-istio-token\") pod \"ec813d98-6cb8-41d1-b947-555d11506c24\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") "
Jul 10 23:33:38.825094 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.824927    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-workload-certs\") pod \"ec813d98-6cb8-41d1-b947-555d11506c24\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") "
Jul 10 23:33:38.825094 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.824951    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-credential-socket\") pod \"ec813d98-6cb8-41d1-b947-555d11506c24\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") "
Jul 10 23:33:38.825094 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.824975    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-istio-envoy\") pod \"ec813d98-6cb8-41d1-b947-555d11506c24\" (UID: \"ec813d98-6cb8-41d1-b947-555d11506c24\") "
Jul 10 23:33:38.825303 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.825097    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ec813d98-6cb8-41d1-b947-555d11506c24" (UID: "ec813d98-6cb8-41d1-b947-555d11506c24"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:33:38.825303 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.825289    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-workload-socket\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:38.825454 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.825329    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ec813d98-6cb8-41d1-b947-555d11506c24" (UID: "ec813d98-6cb8-41d1-b947-555d11506c24"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:33:38.825454 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.825343    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ec813d98-6cb8-41d1-b947-555d11506c24" (UID: "ec813d98-6cb8-41d1-b947-555d11506c24"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:33:38.825802 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.825771    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-istio-data" (OuterVolumeSpecName: "istio-data") pod "ec813d98-6cb8-41d1-b947-555d11506c24" (UID: "ec813d98-6cb8-41d1-b947-555d11506c24"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:33:38.825887 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.825820    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec813d98-6cb8-41d1-b947-555d11506c24-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ec813d98-6cb8-41d1-b947-555d11506c24" (UID: "ec813d98-6cb8-41d1-b947-555d11506c24"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:33:38.827321 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.827289    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ec813d98-6cb8-41d1-b947-555d11506c24-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ec813d98-6cb8-41d1-b947-555d11506c24" (UID: "ec813d98-6cb8-41d1-b947-555d11506c24"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 23:33:38.828035 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.828012    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec813d98-6cb8-41d1-b947-555d11506c24-istio-token" (OuterVolumeSpecName: "istio-token") pod "ec813d98-6cb8-41d1-b947-555d11506c24" (UID: "ec813d98-6cb8-41d1-b947-555d11506c24"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:33:38.828136 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.828093    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ec813d98-6cb8-41d1-b947-555d11506c24" (UID: "ec813d98-6cb8-41d1-b947-555d11506c24"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:33:38.828136 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.828103    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec813d98-6cb8-41d1-b947-555d11506c24-kube-api-access-2wdjv" (OuterVolumeSpecName: "kube-api-access-2wdjv") pod "ec813d98-6cb8-41d1-b947-555d11506c24" (UID: "ec813d98-6cb8-41d1-b947-555d11506c24"). InnerVolumeSpecName "kube-api-access-2wdjv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:33:38.926316 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.926277    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec813d98-6cb8-41d1-b947-555d11506c24-istio-token\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:38.926316 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.926310    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-workload-certs\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:38.926316 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.926318    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-credential-socket\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:38.926316 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.926328    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-istio-envoy\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:38.926664 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.926336    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec813d98-6cb8-41d1-b947-555d11506c24-istiod-ca-cert\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:38.926664 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.926344    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec813d98-6cb8-41d1-b947-555d11506c24-istio-data\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:38.926664 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.926351    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec813d98-6cb8-41d1-b947-555d11506c24-istio-podinfo\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:38.926664 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:38.926384    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wdjv\" (UniqueName: \"kubernetes.io/projected/ec813d98-6cb8-41d1-b947-555d11506c24-kube-api-access-2wdjv\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:39.635628 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:39.635591    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"
Jul 10 23:33:39.636138 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:39.635624    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw" event={"ID":"ec813d98-6cb8-41d1-b947-555d11506c24","Type":"ContainerDied","Data":"2fcc27b9329a21fc84a0d4a298e7eaf5fc0290bea2a12ecbd90b6babf59a5ba1"}
Jul 10 23:33:39.636138 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:39.635692    2563 scope.go:117] "RemoveContainer" containerID="793b6844efc03dd1d03113e00361ce37860ab087abfa4ca067f3cfa2633792e7"
Jul 10 23:33:39.653488 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:39.653442    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"]
Jul 10 23:33:39.657652 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:39.657624    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-7b4cfe-openshift-default-6b8cd98bd5-xdmbw"]
Jul 10 23:33:39.917887 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:39.917801    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:39.978605 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:39.978575    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:40.160332 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:40.160295    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"]
Jul 10 23:33:40.178350 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:40.178269    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:40.402654 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:40.402615    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:40.640895 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:40.640854    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5" podUID="e76fa989-900e-47e1-9470-678dc7c46d4d" containerName="istio-proxy" containerID="cri-o://0fa6cb8ff63aa0ba6f04c5b877b81b678e8b3ab40475dd5d96bad5cd88d5d5b0" gracePeriod=30
Jul 10 23:33:40.753238 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:40.753207    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:41.022528 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.022485    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"]
Jul 10 23:33:41.022876 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.022864    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec813d98-6cb8-41d1-b947-555d11506c24" containerName="istio-proxy"
Jul 10 23:33:41.022932 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.022877    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec813d98-6cb8-41d1-b947-555d11506c24" containerName="istio-proxy"
Jul 10 23:33:41.022932 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.022895    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec813d98-6cb8-41d1-b947-555d11506c24" containerName="istio-proxy"
Jul 10 23:33:41.022932 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.022902    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec813d98-6cb8-41d1-b947-555d11506c24" containerName="istio-proxy"
Jul 10 23:33:41.022932 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.022910    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1877470f-95e9-48d2-a963-787e2f3bd2e1" containerName="maas-api"
Jul 10 23:33:41.022932 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.022915    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1877470f-95e9-48d2-a963-787e2f3bd2e1" containerName="maas-api"
Jul 10 23:33:41.023108 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.022974    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec813d98-6cb8-41d1-b947-555d11506c24" containerName="istio-proxy"
Jul 10 23:33:41.023108 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.022983    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec813d98-6cb8-41d1-b947-555d11506c24" containerName="istio-proxy"
Jul 10 23:33:41.023108 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.022989    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1877470f-95e9-48d2-a963-787e2f3bd2e1" containerName="maas-api"
Jul 10 23:33:41.028008 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.027977    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.030476 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.030453    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-f5848c-openshift-default-dockercfg-5sbs8\""
Jul 10 23:33:41.037867 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.037838    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"]
Jul 10 23:33:41.048719 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.048683    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:41.149949 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.149901    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/906d9f75-9dfe-4014-b699-22e47c5c0c40-istio-token\") pod \"e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") " pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.150166 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.149966    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/906d9f75-9dfe-4014-b699-22e47c5c0c40-istio-podinfo\") pod \"e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") " pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.150166 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.150008    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-credential-socket\") pod \"e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") " pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.150166 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.150026    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/906d9f75-9dfe-4014-b699-22e47c5c0c40-istiod-ca-cert\") pod \"e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") " pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.150166 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.150050    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-workload-socket\") pod \"e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") " pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.150166 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.150085    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-istio-data\") pod \"e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") " pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.150166 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.150142    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-istio-envoy\") pod \"e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") " pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.150166 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.150165    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj6jq\" (UniqueName: \"kubernetes.io/projected/906d9f75-9dfe-4014-b699-22e47c5c0c40-kube-api-access-fj6jq\") pod \"e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") " pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.150575 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.150218    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-workload-certs\") pod \"e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") " pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.251708 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.251650    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/906d9f75-9dfe-4014-b699-22e47c5c0c40-istio-podinfo\") pod \"e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") " pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.251708 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.251727    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-credential-socket\") pod \"e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") " pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.252006 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.251746    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/906d9f75-9dfe-4014-b699-22e47c5c0c40-istiod-ca-cert\") pod \"e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") " pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.252006 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.251764    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-workload-socket\") pod \"e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") " pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.252006 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.251781    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-istio-data\") pod \"e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") " pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.252006 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.251804    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-istio-envoy\") pod \"e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") " pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.252006 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.251824    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj6jq\" (UniqueName: \"kubernetes.io/projected/906d9f75-9dfe-4014-b699-22e47c5c0c40-kube-api-access-fj6jq\") pod \"e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") " pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.252006 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.251855    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-workload-certs\") pod \"e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") " pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.252006 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.251919    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/906d9f75-9dfe-4014-b699-22e47c5c0c40-istio-token\") pod \"e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") " pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.252410 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.252233    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-credential-socket\") pod \"e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") " pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.252410 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.252285    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-workload-socket\") pod \"e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") " pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.252521 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.252474    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-istio-data\") pod \"e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") " pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.252576 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.252521    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-workload-certs\") pod \"e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") " pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.252740 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.252722    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/906d9f75-9dfe-4014-b699-22e47c5c0c40-istiod-ca-cert\") pod \"e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") " pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.254602 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.254568    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-istio-envoy\") pod \"e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") " pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.254602 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.254584    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/906d9f75-9dfe-4014-b699-22e47c5c0c40-istio-podinfo\") pod \"e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") " pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.259891 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.259864    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/906d9f75-9dfe-4014-b699-22e47c5c0c40-istio-token\") pod \"e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") " pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.260070 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.260047    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj6jq\" (UniqueName: \"kubernetes.io/projected/906d9f75-9dfe-4014-b699-22e47c5c0c40-kube-api-access-fj6jq\") pod \"e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") " pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.340165 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.340064    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:41.488305 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.488275    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"]
Jul 10 23:33:41.490087 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:33:41.490052    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod906d9f75_9dfe_4014_b699_22e47c5c0c40.slice/crio-22eda82cb5c57a7157e65c9c213b7dec45d2485418b253c27d51193edcdd065f WatchSource:0}: Error finding container 22eda82cb5c57a7157e65c9c213b7dec45d2485418b253c27d51193edcdd065f: Status 404 returned error can't find the container with id 22eda82cb5c57a7157e65c9c213b7dec45d2485418b253c27d51193edcdd065f
Jul 10 23:33:41.492436 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.492403    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:33:41.492526 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.492466    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:33:41.492526 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.492493    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:33:41.557104 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.557075    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec813d98-6cb8-41d1-b947-555d11506c24" path="/var/lib/kubelet/pods/ec813d98-6cb8-41d1-b947-555d11506c24/volumes"
Jul 10 23:33:41.649321 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.649230    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp" event={"ID":"906d9f75-9dfe-4014-b699-22e47c5c0c40","Type":"ContainerStarted","Data":"430deb35447f03394fd35ae35815fa70ef9034c489a2b1061fe0a72f5c768125"}
Jul 10 23:33:41.649321 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.649296    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp" event={"ID":"906d9f75-9dfe-4014-b699-22e47c5c0c40","Type":"ContainerStarted","Data":"22eda82cb5c57a7157e65c9c213b7dec45d2485418b253c27d51193edcdd065f"}
Jul 10 23:33:41.667531 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.667443    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp" podStartSLOduration=0.667419324 podStartE2EDuration="667.419324ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:33:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:33:41.666434182 +0000 UTC m=+2282.553170326" watchObservedRunningTime="2026-07-10 23:33:41.667419324 +0000 UTC m=+2282.554155467"
Jul 10 23:33:41.787185 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:41.787152    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:42.340226 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:42.340185    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:42.345410 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:42.345377    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:42.654135 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:42.654048    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:42.655169 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:42.655149    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:33:43.706143 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:43.706109    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:43.804478 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:43.804445    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:43.902946 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:43.902909    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:44.003607 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:44.003573    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:44.426903 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:44.426827    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:44.529574 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:44.529528    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:45.899880 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:45.899853    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:45.996537 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:45.996496    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e76fa989-900e-47e1-9470-678dc7c46d4d-istio-podinfo\") pod \"e76fa989-900e-47e1-9470-678dc7c46d4d\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") "
Jul 10 23:33:45.996721 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:45.996570    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-workload-certs\") pod \"e76fa989-900e-47e1-9470-678dc7c46d4d\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") "
Jul 10 23:33:45.996721 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:45.996600    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-istio-data\") pod \"e76fa989-900e-47e1-9470-678dc7c46d4d\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") "
Jul 10 23:33:45.996721 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:45.996619    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7dz9\" (UniqueName: \"kubernetes.io/projected/e76fa989-900e-47e1-9470-678dc7c46d4d-kube-api-access-w7dz9\") pod \"e76fa989-900e-47e1-9470-678dc7c46d4d\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") "
Jul 10 23:33:45.996721 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:45.996654    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-credential-socket\") pod \"e76fa989-900e-47e1-9470-678dc7c46d4d\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") "
Jul 10 23:33:45.996721 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:45.996692    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e76fa989-900e-47e1-9470-678dc7c46d4d-istiod-ca-cert\") pod \"e76fa989-900e-47e1-9470-678dc7c46d4d\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") "
Jul 10 23:33:45.996990 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:45.996720    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-istio-envoy\") pod \"e76fa989-900e-47e1-9470-678dc7c46d4d\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") "
Jul 10 23:33:45.996990 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:45.996750    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-workload-socket\") pod \"e76fa989-900e-47e1-9470-678dc7c46d4d\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") "
Jul 10 23:33:45.996990 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:45.996786    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e76fa989-900e-47e1-9470-678dc7c46d4d-istio-token\") pod \"e76fa989-900e-47e1-9470-678dc7c46d4d\" (UID: \"e76fa989-900e-47e1-9470-678dc7c46d4d\") "
Jul 10 23:33:45.996990 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:45.996976    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e76fa989-900e-47e1-9470-678dc7c46d4d" (UID: "e76fa989-900e-47e1-9470-678dc7c46d4d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:33:45.997209 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:45.997099    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-workload-certs\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:45.997209 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:45.997117    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e76fa989-900e-47e1-9470-678dc7c46d4d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e76fa989-900e-47e1-9470-678dc7c46d4d" (UID: "e76fa989-900e-47e1-9470-678dc7c46d4d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:33:45.997333 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:45.997307    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e76fa989-900e-47e1-9470-678dc7c46d4d" (UID: "e76fa989-900e-47e1-9470-678dc7c46d4d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:33:45.997540 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:45.997517    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e76fa989-900e-47e1-9470-678dc7c46d4d" (UID: "e76fa989-900e-47e1-9470-678dc7c46d4d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:33:45.997797 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:45.997772    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-istio-data" (OuterVolumeSpecName: "istio-data") pod "e76fa989-900e-47e1-9470-678dc7c46d4d" (UID: "e76fa989-900e-47e1-9470-678dc7c46d4d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:33:45.999126 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:45.999102    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e76fa989-900e-47e1-9470-678dc7c46d4d-istio-token" (OuterVolumeSpecName: "istio-token") pod "e76fa989-900e-47e1-9470-678dc7c46d4d" (UID: "e76fa989-900e-47e1-9470-678dc7c46d4d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:33:45.999542 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:45.999513    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e76fa989-900e-47e1-9470-678dc7c46d4d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e76fa989-900e-47e1-9470-678dc7c46d4d" (UID: "e76fa989-900e-47e1-9470-678dc7c46d4d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 23:33:45.999542 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:45.999517    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e76fa989-900e-47e1-9470-678dc7c46d4d-kube-api-access-w7dz9" (OuterVolumeSpecName: "kube-api-access-w7dz9") pod "e76fa989-900e-47e1-9470-678dc7c46d4d" (UID: "e76fa989-900e-47e1-9470-678dc7c46d4d"). InnerVolumeSpecName "kube-api-access-w7dz9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:33:45.999665 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:45.999594    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e76fa989-900e-47e1-9470-678dc7c46d4d" (UID: "e76fa989-900e-47e1-9470-678dc7c46d4d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:33:46.097878 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:46.097835    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-istio-data\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:46.097878 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:46.097872    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7dz9\" (UniqueName: \"kubernetes.io/projected/e76fa989-900e-47e1-9470-678dc7c46d4d-kube-api-access-w7dz9\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:46.097878 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:46.097881    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-credential-socket\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:46.098126 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:46.097890    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e76fa989-900e-47e1-9470-678dc7c46d4d-istiod-ca-cert\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:46.098126 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:46.097899    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-istio-envoy\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:46.098126 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:46.097907    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e76fa989-900e-47e1-9470-678dc7c46d4d-workload-socket\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:46.098126 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:46.097915    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e76fa989-900e-47e1-9470-678dc7c46d4d-istio-token\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:46.098126 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:46.097922    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e76fa989-900e-47e1-9470-678dc7c46d4d-istio-podinfo\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:33:46.674999 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:46.674961    2563 generic.go:356] "Generic (PLEG): container finished" podID="e76fa989-900e-47e1-9470-678dc7c46d4d" containerID="0fa6cb8ff63aa0ba6f04c5b877b81b678e8b3ab40475dd5d96bad5cd88d5d5b0" exitCode=0
Jul 10 23:33:46.675199 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:46.675024    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"
Jul 10 23:33:46.675199 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:46.675046    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5" event={"ID":"e76fa989-900e-47e1-9470-678dc7c46d4d","Type":"ContainerDied","Data":"0fa6cb8ff63aa0ba6f04c5b877b81b678e8b3ab40475dd5d96bad5cd88d5d5b0"}
Jul 10 23:33:46.675199 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:46.675098    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5" event={"ID":"e76fa989-900e-47e1-9470-678dc7c46d4d","Type":"ContainerDied","Data":"6d5205c34a25cf93051162e056a9ef5c1bf0bda0168eeeebab7b87ffc9a67ed2"}
Jul 10 23:33:46.675199 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:46.675119    2563 scope.go:117] "RemoveContainer" containerID="0fa6cb8ff63aa0ba6f04c5b877b81b678e8b3ab40475dd5d96bad5cd88d5d5b0"
Jul 10 23:33:46.685025 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:46.684992    2563 scope.go:117] "RemoveContainer" containerID="67e73a4754a4e7a8238e8e16d9923deea7ceb6901ebbbcda36440324308b76d2"
Jul 10 23:33:46.693986 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:46.693944    2563 scope.go:117] "RemoveContainer" containerID="0fa6cb8ff63aa0ba6f04c5b877b81b678e8b3ab40475dd5d96bad5cd88d5d5b0"
Jul 10 23:33:46.694515 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:33:46.694483    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fa6cb8ff63aa0ba6f04c5b877b81b678e8b3ab40475dd5d96bad5cd88d5d5b0\": container with ID starting with 0fa6cb8ff63aa0ba6f04c5b877b81b678e8b3ab40475dd5d96bad5cd88d5d5b0 not found: ID does not exist" containerID="0fa6cb8ff63aa0ba6f04c5b877b81b678e8b3ab40475dd5d96bad5cd88d5d5b0"
Jul 10 23:33:46.694621 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:46.694531    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fa6cb8ff63aa0ba6f04c5b877b81b678e8b3ab40475dd5d96bad5cd88d5d5b0"} err="failed to get container status \"0fa6cb8ff63aa0ba6f04c5b877b81b678e8b3ab40475dd5d96bad5cd88d5d5b0\": rpc error: code = NotFound desc = could not find container \"0fa6cb8ff63aa0ba6f04c5b877b81b678e8b3ab40475dd5d96bad5cd88d5d5b0\": container with ID starting with 0fa6cb8ff63aa0ba6f04c5b877b81b678e8b3ab40475dd5d96bad5cd88d5d5b0 not found: ID does not exist"
Jul 10 23:33:46.694621 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:46.694556    2563 scope.go:117] "RemoveContainer" containerID="67e73a4754a4e7a8238e8e16d9923deea7ceb6901ebbbcda36440324308b76d2"
Jul 10 23:33:46.694903 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:33:46.694871    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67e73a4754a4e7a8238e8e16d9923deea7ceb6901ebbbcda36440324308b76d2\": container with ID starting with 67e73a4754a4e7a8238e8e16d9923deea7ceb6901ebbbcda36440324308b76d2 not found: ID does not exist" containerID="67e73a4754a4e7a8238e8e16d9923deea7ceb6901ebbbcda36440324308b76d2"
Jul 10 23:33:46.694956 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:46.694912    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67e73a4754a4e7a8238e8e16d9923deea7ceb6901ebbbcda36440324308b76d2"} err="failed to get container status \"67e73a4754a4e7a8238e8e16d9923deea7ceb6901ebbbcda36440324308b76d2\": rpc error: code = NotFound desc = could not find container \"67e73a4754a4e7a8238e8e16d9923deea7ceb6901ebbbcda36440324308b76d2\": container with ID starting with 67e73a4754a4e7a8238e8e16d9923deea7ceb6901ebbbcda36440324308b76d2 not found: ID does not exist"
Jul 10 23:33:46.696636 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:46.696616    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"]
Jul 10 23:33:46.699875 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:46.699850    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-38233f-openshift-default-85b85b6699-q5vl5"]
Jul 10 23:33:47.557446 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:47.557411    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e76fa989-900e-47e1-9470-678dc7c46d4d" path="/var/lib/kubelet/pods/e76fa989-900e-47e1-9470-678dc7c46d4d/volumes"
Jul 10 23:33:48.164315 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:48.164273    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:48.515685 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:48.515640    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:48.797731 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:48.797635    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"]
Jul 10 23:33:48.798187 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:48.798044    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e76fa989-900e-47e1-9470-678dc7c46d4d" containerName="istio-proxy"
Jul 10 23:33:48.798187 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:48.798058    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e76fa989-900e-47e1-9470-678dc7c46d4d" containerName="istio-proxy"
Jul 10 23:33:48.798187 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:48.798119    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e76fa989-900e-47e1-9470-678dc7c46d4d" containerName="istio-proxy"
Jul 10 23:33:48.802924 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:48.802898    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:48.805346 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:48.805323    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-27111a-openshift-default-dockercfg-vm2qr\""
Jul 10 23:33:48.813516 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:48.813488    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"]
Jul 10 23:33:48.826630 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:48.826595    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:48.926719 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:48.926680    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-istio-envoy\") pod \"e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") " pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:48.926898 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:48.926729    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-workload-certs\") pod \"e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") " pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:48.926898 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:48.926811    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-istio-data\") pod \"e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") " pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:48.926898 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:48.926850    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-workload-socket\") pod \"e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") " pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:48.926898 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:48.926872    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-credential-socket\") pod \"e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") " pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:48.927053 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:48.926949    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10c229cb-0d97-4fc0-af69-032d293f26af-istio-token\") pod \"e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") " pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:48.927053 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:48.926971    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10c229cb-0d97-4fc0-af69-032d293f26af-istiod-ca-cert\") pod \"e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") " pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:48.927053 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:48.927001    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j55sp\" (UniqueName: \"kubernetes.io/projected/10c229cb-0d97-4fc0-af69-032d293f26af-kube-api-access-j55sp\") pod \"e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") " pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:48.927053 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:48.927042    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10c229cb-0d97-4fc0-af69-032d293f26af-istio-podinfo\") pod \"e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") " pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:49.028397 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:49.028316    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-istio-envoy\") pod \"e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") " pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:49.028607 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:49.028413    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-workload-certs\") pod \"e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") " pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:49.028607 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:49.028441    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-istio-data\") pod \"e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") " pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:49.028607 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:49.028462    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-workload-socket\") pod \"e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") " pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:49.028607 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:49.028496    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-credential-socket\") pod \"e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") " pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:49.028607 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:49.028575    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10c229cb-0d97-4fc0-af69-032d293f26af-istio-token\") pod \"e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") " pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:49.028607 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:49.028602    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10c229cb-0d97-4fc0-af69-032d293f26af-istiod-ca-cert\") pod \"e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") " pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:49.028928 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:49.028646    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j55sp\" (UniqueName: \"kubernetes.io/projected/10c229cb-0d97-4fc0-af69-032d293f26af-kube-api-access-j55sp\") pod \"e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") " pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:49.028928 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:49.028715    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10c229cb-0d97-4fc0-af69-032d293f26af-istio-podinfo\") pod \"e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") " pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:49.028928 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:49.028845    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-istio-data\") pod \"e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") " pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:49.029087 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:49.029021    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-credential-socket\") pod \"e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") " pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:49.029264 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:49.029240    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-workload-socket\") pod \"e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") " pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:49.029398 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:49.029386    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-workload-certs\") pod \"e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") " pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:49.029534 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:49.029391    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10c229cb-0d97-4fc0-af69-032d293f26af-istiod-ca-cert\") pod \"e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") " pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:49.030929 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:49.030902    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-istio-envoy\") pod \"e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") " pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:49.031321 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:49.031300    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10c229cb-0d97-4fc0-af69-032d293f26af-istio-podinfo\") pod \"e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") " pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:49.036745 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:49.036710    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j55sp\" (UniqueName: \"kubernetes.io/projected/10c229cb-0d97-4fc0-af69-032d293f26af-kube-api-access-j55sp\") pod \"e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") " pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:49.036903 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:49.036883    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10c229cb-0d97-4fc0-af69-032d293f26af-istio-token\") pod \"e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") " pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:49.116172 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:49.116070    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:49.264886 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:49.264855    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"]
Jul 10 23:33:49.267440 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:33:49.267416    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10c229cb_0d97_4fc0_af69_032d293f26af.slice/crio-131ebc1eff29c6f6c9c44f1089491ac530c11a8bf98a8a15e84165016ed80670 WatchSource:0}: Error finding container 131ebc1eff29c6f6c9c44f1089491ac530c11a8bf98a8a15e84165016ed80670: Status 404 returned error can't find the container with id 131ebc1eff29c6f6c9c44f1089491ac530c11a8bf98a8a15e84165016ed80670
Jul 10 23:33:49.270162 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:49.270127    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:33:49.270272 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:49.270211    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:33:49.270272 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:49.270258    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:33:49.504903 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:49.504870    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:49.697646 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:49.697609    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw" event={"ID":"10c229cb-0d97-4fc0-af69-032d293f26af","Type":"ContainerStarted","Data":"18c22abc8d3b260cec80a80c4999126e0988d2188f32b1154aacd35c6828e3e8"}
Jul 10 23:33:49.697798 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:49.697667    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw" event={"ID":"10c229cb-0d97-4fc0-af69-032d293f26af","Type":"ContainerStarted","Data":"131ebc1eff29c6f6c9c44f1089491ac530c11a8bf98a8a15e84165016ed80670"}
Jul 10 23:33:49.718273 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:49.718220    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw" podStartSLOduration=1.7182036090000001 podStartE2EDuration="1.718203609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:33:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:33:49.71603151 +0000 UTC m=+2290.602767655" watchObservedRunningTime="2026-07-10 23:33:49.718203609 +0000 UTC m=+2290.604939825"
Jul 10 23:33:50.116243 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:50.116206    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:50.121746 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:50.121713    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:50.702869 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:50.702834    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:50.703868 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:50.703845    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:33:51.645710 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:51.645675    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:51.749561 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:51.749528    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:51.817295 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:51.817256    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-df5db5888-sbmq9"]
Jul 10 23:33:51.817665 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:51.817652    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e76fa989-900e-47e1-9470-678dc7c46d4d" containerName="istio-proxy"
Jul 10 23:33:51.817718 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:51.817666    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e76fa989-900e-47e1-9470-678dc7c46d4d" containerName="istio-proxy"
Jul 10 23:33:51.817757 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:51.817729    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e76fa989-900e-47e1-9470-678dc7c46d4d" containerName="istio-proxy"
Jul 10 23:33:51.820956 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:51.820936    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-df5db5888-sbmq9"
Jul 10 23:33:51.823666 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:51.823645    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-27111a-serving-cert\""
Jul 10 23:33:51.823666 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:51.823656    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fxg5k\""
Jul 10 23:33:51.823844 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:51.823667    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 10 23:33:51.824547 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:51.824527    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 23:33:51.831247 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:51.831221    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-df5db5888-sbmq9"]
Jul 10 23:33:51.856390 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:51.856331    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e7ab61b-0379-445c-8080-538f710aedaf-openshift-service-ca\") pod \"maas-api-e2e-shared-b-27111a-df5db5888-sbmq9\" (UID: \"3e7ab61b-0379-445c-8080-538f710aedaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-df5db5888-sbmq9"
Jul 10 23:33:51.856589 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:51.856438    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e7ab61b-0379-445c-8080-538f710aedaf-maas-api-tls\") pod \"maas-api-e2e-shared-b-27111a-df5db5888-sbmq9\" (UID: \"3e7ab61b-0379-445c-8080-538f710aedaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-df5db5888-sbmq9"
Jul 10 23:33:51.856589 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:51.856517    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc22b\" (UniqueName: \"kubernetes.io/projected/3e7ab61b-0379-445c-8080-538f710aedaf-kube-api-access-nc22b\") pod \"maas-api-e2e-shared-b-27111a-df5db5888-sbmq9\" (UID: \"3e7ab61b-0379-445c-8080-538f710aedaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-df5db5888-sbmq9"
Jul 10 23:33:51.913836 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:51.913700    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:51.957771 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:51.957728    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e7ab61b-0379-445c-8080-538f710aedaf-openshift-service-ca\") pod \"maas-api-e2e-shared-b-27111a-df5db5888-sbmq9\" (UID: \"3e7ab61b-0379-445c-8080-538f710aedaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-df5db5888-sbmq9"
Jul 10 23:33:51.957964 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:51.957790    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e7ab61b-0379-445c-8080-538f710aedaf-maas-api-tls\") pod \"maas-api-e2e-shared-b-27111a-df5db5888-sbmq9\" (UID: \"3e7ab61b-0379-445c-8080-538f710aedaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-df5db5888-sbmq9"
Jul 10 23:33:51.957964 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:51.957837    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc22b\" (UniqueName: \"kubernetes.io/projected/3e7ab61b-0379-445c-8080-538f710aedaf-kube-api-access-nc22b\") pod \"maas-api-e2e-shared-b-27111a-df5db5888-sbmq9\" (UID: \"3e7ab61b-0379-445c-8080-538f710aedaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-df5db5888-sbmq9"
Jul 10 23:33:51.958041 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:33:51.957980    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-serving-cert: secret "maas-api-e2e-shared-b-27111a-serving-cert" not found
Jul 10 23:33:51.958082 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:33:51.958073    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e7ab61b-0379-445c-8080-538f710aedaf-maas-api-tls podName:3e7ab61b-0379-445c-8080-538f710aedaf nodeName:}" failed. No retries permitted until 2026-07-10 23:33:52.458052525 +0000 UTC m=+2293.344788650 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3e7ab61b-0379-445c-8080-538f710aedaf-maas-api-tls") pod "maas-api-e2e-shared-b-27111a-df5db5888-sbmq9" (UID: "3e7ab61b-0379-445c-8080-538f710aedaf") : secret "maas-api-e2e-shared-b-27111a-serving-cert" not found
Jul 10 23:33:51.958489 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:51.958469    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e7ab61b-0379-445c-8080-538f710aedaf-openshift-service-ca\") pod \"maas-api-e2e-shared-b-27111a-df5db5888-sbmq9\" (UID: \"3e7ab61b-0379-445c-8080-538f710aedaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-df5db5888-sbmq9"
Jul 10 23:33:51.967636 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:51.967611    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc22b\" (UniqueName: \"kubernetes.io/projected/3e7ab61b-0379-445c-8080-538f710aedaf-kube-api-access-nc22b\") pod \"maas-api-e2e-shared-b-27111a-df5db5888-sbmq9\" (UID: \"3e7ab61b-0379-445c-8080-538f710aedaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-df5db5888-sbmq9"
Jul 10 23:33:52.022852 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:52.022814    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:52.335245 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:52.335193    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:52.436112 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:52.436080    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:52.462157 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:52.462122    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e7ab61b-0379-445c-8080-538f710aedaf-maas-api-tls\") pod \"maas-api-e2e-shared-b-27111a-df5db5888-sbmq9\" (UID: \"3e7ab61b-0379-445c-8080-538f710aedaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-df5db5888-sbmq9"
Jul 10 23:33:52.464932 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:52.464896    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e7ab61b-0379-445c-8080-538f710aedaf-maas-api-tls\") pod \"maas-api-e2e-shared-b-27111a-df5db5888-sbmq9\" (UID: \"3e7ab61b-0379-445c-8080-538f710aedaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-df5db5888-sbmq9"
Jul 10 23:33:52.733047 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:52.733003    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-df5db5888-sbmq9"
Jul 10 23:33:52.880544 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:52.880516    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-df5db5888-sbmq9"]
Jul 10 23:33:52.882958 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:33:52.882932    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e7ab61b_0379_445c_8080_538f710aedaf.slice/crio-b2f295af3640b8a214ab7feb15d10a90bba0d3456d3eacf6525b475af589fe5d WatchSource:0}: Error finding container b2f295af3640b8a214ab7feb15d10a90bba0d3456d3eacf6525b475af589fe5d: Status 404 returned error can't find the container with id b2f295af3640b8a214ab7feb15d10a90bba0d3456d3eacf6525b475af589fe5d
Jul 10 23:33:53.721549 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:53.721457    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-df5db5888-sbmq9" event={"ID":"3e7ab61b-0379-445c-8080-538f710aedaf","Type":"ContainerStarted","Data":"1edf4edd4603765b8bb94a7ba80e51eea50012a5af57aebdce44c0a00cc786f1"}
Jul 10 23:33:53.721549 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:53.721508    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-df5db5888-sbmq9" event={"ID":"3e7ab61b-0379-445c-8080-538f710aedaf","Type":"ContainerStarted","Data":"b2f295af3640b8a214ab7feb15d10a90bba0d3456d3eacf6525b475af589fe5d"}
Jul 10 23:33:53.721749 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:53.721595    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-df5db5888-sbmq9"
Jul 10 23:33:53.737708 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:53.737656    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-df5db5888-sbmq9" podStartSLOduration=2.389787741 podStartE2EDuration="2.737637577s" totalImagesPullingTime="347.849836ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:33:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:33:53.735974325 +0000 UTC m=+2294.622710469" watchObservedRunningTime="2026-07-10 23:33:53.737637577 +0000 UTC m=+2294.624373719"
Jul 10 23:33:56.150759 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:56.150722    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:33:58.695553 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:58.695513    2563 patch_prober.go:28] interesting pod/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 10 23:33:58.696024 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:58.695575    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp" podUID="906d9f75-9dfe-4014-b699-22e47c5c0c40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 10 23:33:59.732593 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:33:59.732561    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-df5db5888-sbmq9"
Jul 10 23:34:00.766334 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:00.766293    2563 generic.go:356] "Generic (PLEG): container finished" podID="906d9f75-9dfe-4014-b699-22e47c5c0c40" containerID="430deb35447f03394fd35ae35815fa70ef9034c489a2b1061fe0a72f5c768125" exitCode=0
Jul 10 23:34:00.767128 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:00.766977    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp" event={"ID":"906d9f75-9dfe-4014-b699-22e47c5c0c40","Type":"ContainerDied","Data":"430deb35447f03394fd35ae35815fa70ef9034c489a2b1061fe0a72f5c768125"}
Jul 10 23:34:00.767885 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:00.767868    2563 scope.go:117] "RemoveContainer" containerID="430deb35447f03394fd35ae35815fa70ef9034c489a2b1061fe0a72f5c768125"
Jul 10 23:34:00.769992 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:00.769612    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:34:00.769992 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:00.769736    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:34:00.769992 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:00.769786    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:34:01.777580 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:01.777544    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp" event={"ID":"906d9f75-9dfe-4014-b699-22e47c5c0c40","Type":"ContainerStarted","Data":"a1365c51c163fc0610414033b85ddf3bed5fe1cf9bcdd86a467ff911c8fce412"}
Jul 10 23:34:02.340575 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:02.340531    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:34:02.340778 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:02.340751    2563 patch_prober.go:28] interesting pod/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 10 23:34:02.340839 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:02.340806    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp" podUID="906d9f75-9dfe-4014-b699-22e47c5c0c40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 10 23:34:03.340492 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:03.340453    2563 patch_prober.go:28] interesting pod/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 10 23:34:03.340893 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:03.340511    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp" podUID="906d9f75-9dfe-4014-b699-22e47c5c0c40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 10 23:34:04.341424 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:04.341382    2563 patch_prober.go:28] interesting pod/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 10 23:34:04.341843 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:04.341438    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp" podUID="906d9f75-9dfe-4014-b699-22e47c5c0c40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 10 23:34:05.340721 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:05.340682    2563 patch_prober.go:28] interesting pod/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 10 23:34:05.340914 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:05.340741    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp" podUID="906d9f75-9dfe-4014-b699-22e47c5c0c40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 10 23:34:06.038468 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:06.038409    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:34:06.139059 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:06.139021    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:34:06.344535 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:06.344450    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:34:06.344778 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:06.344747    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:34:06.345806 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:06.345784    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:34:06.541826 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:06.541792    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:34:06.745606 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:06.745568    2563 patch_prober.go:28] interesting pod/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 10 23:34:06.745810 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:06.745641    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw" podUID="10c229cb-0d97-4fc0-af69-032d293f26af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 10 23:34:08.821877 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:08.821836    2563 generic.go:356] "Generic (PLEG): container finished" podID="10c229cb-0d97-4fc0-af69-032d293f26af" containerID="18c22abc8d3b260cec80a80c4999126e0988d2188f32b1154aacd35c6828e3e8" exitCode=0
Jul 10 23:34:08.822402 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:08.821913    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw" event={"ID":"10c229cb-0d97-4fc0-af69-032d293f26af","Type":"ContainerDied","Data":"18c22abc8d3b260cec80a80c4999126e0988d2188f32b1154aacd35c6828e3e8"}
Jul 10 23:34:08.822402 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:08.822385    2563 scope.go:117] "RemoveContainer" containerID="18c22abc8d3b260cec80a80c4999126e0988d2188f32b1154aacd35c6828e3e8"
Jul 10 23:34:08.823478 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:08.823449    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:34:08.823541 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:08.823520    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:34:08.823578 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:08.823551    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 10 23:34:09.829466 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:09.829425    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw" event={"ID":"10c229cb-0d97-4fc0-af69-032d293f26af","Type":"ContainerStarted","Data":"37c716a1bc583d9099b8cfceef72b7f5dcaaa42b399b68dfd92491c207b7f805"}
Jul 10 23:34:10.116306 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:10.116198    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:34:10.116550 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:10.116522    2563 patch_prober.go:28] interesting pod/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 10 23:34:10.116642 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:10.116578    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw" podUID="10c229cb-0d97-4fc0-af69-032d293f26af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 10 23:34:11.116642 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:11.116602    2563 patch_prober.go:28] interesting pod/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 10 23:34:11.117044 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:11.116673    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw" podUID="10c229cb-0d97-4fc0-af69-032d293f26af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 10 23:34:12.116660 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:12.116619    2563 patch_prober.go:28] interesting pod/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 10 23:34:12.117152 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:12.116700    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw" podUID="10c229cb-0d97-4fc0-af69-032d293f26af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 10 23:34:13.116584 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:13.116539    2563 patch_prober.go:28] interesting pod/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 10 23:34:13.116800 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:13.116602    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw" podUID="10c229cb-0d97-4fc0-af69-032d293f26af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 10 23:34:13.950123 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:13.950083    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:34:14.049664 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:14.049629    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:34:14.126781 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:14.126750    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:34:14.127221 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:14.127119    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:34:14.127876 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:14.127856    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:34:39.023869 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.023782    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"]
Jul 10 23:34:39.027822 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.027798    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"
Jul 10 23:34:39.031083 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.031055    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-27111a\"/\"default-dockercfg-9hj8q\""
Jul 10 23:34:39.031083 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.031075    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-27111a\"/\"openshift-service-ca.crt\""
Jul 10 23:34:39.031274 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.031082    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-27111a\"/\"kube-root-ca.crt\""
Jul 10 23:34:39.031274 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.031060    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-27111a\"/\"auth-test-model-27111a-kserve-self-signed-certs\""
Jul 10 23:34:39.036020 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.035997    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"]
Jul 10 23:34:39.112105 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.112060    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:34:39.123282 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.123247    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxlfq\" (UniqueName: \"kubernetes.io/projected/4b94ad69-e327-4dee-b291-b3f808132bde-kube-api-access-kxlfq\") pod \"auth-test-model-27111a-kserve-7448ffd8d7-n2lj5\" (UID: \"4b94ad69-e327-4dee-b291-b3f808132bde\") " pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"
Jul 10 23:34:39.123475 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.123356    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b94ad69-e327-4dee-b291-b3f808132bde-tls-certs\") pod \"auth-test-model-27111a-kserve-7448ffd8d7-n2lj5\" (UID: \"4b94ad69-e327-4dee-b291-b3f808132bde\") " pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"
Jul 10 23:34:39.123475 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.123409    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b94ad69-e327-4dee-b291-b3f808132bde-home\") pod \"auth-test-model-27111a-kserve-7448ffd8d7-n2lj5\" (UID: \"4b94ad69-e327-4dee-b291-b3f808132bde\") " pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"
Jul 10 23:34:39.123475 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.123455    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b94ad69-e327-4dee-b291-b3f808132bde-model-cache\") pod \"auth-test-model-27111a-kserve-7448ffd8d7-n2lj5\" (UID: \"4b94ad69-e327-4dee-b291-b3f808132bde\") " pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"
Jul 10 23:34:39.123587 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.123514    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b94ad69-e327-4dee-b291-b3f808132bde-tmp-dir\") pod \"auth-test-model-27111a-kserve-7448ffd8d7-n2lj5\" (UID: \"4b94ad69-e327-4dee-b291-b3f808132bde\") " pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"
Jul 10 23:34:39.123587 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.123573    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b94ad69-e327-4dee-b291-b3f808132bde-dshm\") pod \"auth-test-model-27111a-kserve-7448ffd8d7-n2lj5\" (UID: \"4b94ad69-e327-4dee-b291-b3f808132bde\") " pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"
Jul 10 23:34:39.224589 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.224550    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b94ad69-e327-4dee-b291-b3f808132bde-model-cache\") pod \"auth-test-model-27111a-kserve-7448ffd8d7-n2lj5\" (UID: \"4b94ad69-e327-4dee-b291-b3f808132bde\") " pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"
Jul 10 23:34:39.224807 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.224602    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b94ad69-e327-4dee-b291-b3f808132bde-tmp-dir\") pod \"auth-test-model-27111a-kserve-7448ffd8d7-n2lj5\" (UID: \"4b94ad69-e327-4dee-b291-b3f808132bde\") " pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"
Jul 10 23:34:39.224807 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.224641    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b94ad69-e327-4dee-b291-b3f808132bde-dshm\") pod \"auth-test-model-27111a-kserve-7448ffd8d7-n2lj5\" (UID: \"4b94ad69-e327-4dee-b291-b3f808132bde\") " pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"
Jul 10 23:34:39.224807 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.224675    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxlfq\" (UniqueName: \"kubernetes.io/projected/4b94ad69-e327-4dee-b291-b3f808132bde-kube-api-access-kxlfq\") pod \"auth-test-model-27111a-kserve-7448ffd8d7-n2lj5\" (UID: \"4b94ad69-e327-4dee-b291-b3f808132bde\") " pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"
Jul 10 23:34:39.224807 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.224742    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b94ad69-e327-4dee-b291-b3f808132bde-tls-certs\") pod \"auth-test-model-27111a-kserve-7448ffd8d7-n2lj5\" (UID: \"4b94ad69-e327-4dee-b291-b3f808132bde\") " pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"
Jul 10 23:34:39.224807 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.224778    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b94ad69-e327-4dee-b291-b3f808132bde-home\") pod \"auth-test-model-27111a-kserve-7448ffd8d7-n2lj5\" (UID: \"4b94ad69-e327-4dee-b291-b3f808132bde\") " pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"
Jul 10 23:34:39.225193 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.225169    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b94ad69-e327-4dee-b291-b3f808132bde-tmp-dir\") pod \"auth-test-model-27111a-kserve-7448ffd8d7-n2lj5\" (UID: \"4b94ad69-e327-4dee-b291-b3f808132bde\") " pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"
Jul 10 23:34:39.225537 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.225505    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b94ad69-e327-4dee-b291-b3f808132bde-model-cache\") pod \"auth-test-model-27111a-kserve-7448ffd8d7-n2lj5\" (UID: \"4b94ad69-e327-4dee-b291-b3f808132bde\") " pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"
Jul 10 23:34:39.225774 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.225663    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b94ad69-e327-4dee-b291-b3f808132bde-home\") pod \"auth-test-model-27111a-kserve-7448ffd8d7-n2lj5\" (UID: \"4b94ad69-e327-4dee-b291-b3f808132bde\") " pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"
Jul 10 23:34:39.227810 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.227783    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b94ad69-e327-4dee-b291-b3f808132bde-tls-certs\") pod \"auth-test-model-27111a-kserve-7448ffd8d7-n2lj5\" (UID: \"4b94ad69-e327-4dee-b291-b3f808132bde\") " pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"
Jul 10 23:34:39.228413 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.228385    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b94ad69-e327-4dee-b291-b3f808132bde-dshm\") pod \"auth-test-model-27111a-kserve-7448ffd8d7-n2lj5\" (UID: \"4b94ad69-e327-4dee-b291-b3f808132bde\") " pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"
Jul 10 23:34:39.233338 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.233302    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxlfq\" (UniqueName: \"kubernetes.io/projected/4b94ad69-e327-4dee-b291-b3f808132bde-kube-api-access-kxlfq\") pod \"auth-test-model-27111a-kserve-7448ffd8d7-n2lj5\" (UID: \"4b94ad69-e327-4dee-b291-b3f808132bde\") " pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"
Jul 10 23:34:39.339441 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.339306    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"
Jul 10 23:34:39.502537 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.502475    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"]
Jul 10 23:34:39.505005 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:34:39.504967    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b94ad69_e327_4dee_b291_b3f808132bde.slice/crio-11afb7995136151740e24590fe7c69de803407443c55f71614c79edea74c47e0 WatchSource:0}: Error finding container 11afb7995136151740e24590fe7c69de803407443c55f71614c79edea74c47e0: Status 404 returned error can't find the container with id 11afb7995136151740e24590fe7c69de803407443c55f71614c79edea74c47e0
Jul 10 23:34:39.996037 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.995996    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5" event={"ID":"4b94ad69-e327-4dee-b291-b3f808132bde","Type":"ContainerStarted","Data":"5e7497a1535d983e4bedb744f7de85f7b1acf49b833f46e1e5c303e071305cd7"}
Jul 10 23:34:39.996213 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:39.996045    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5" event={"ID":"4b94ad69-e327-4dee-b291-b3f808132bde","Type":"ContainerStarted","Data":"11afb7995136151740e24590fe7c69de803407443c55f71614c79edea74c47e0"}
Jul 10 23:34:40.011900 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:40.011847    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5" podStartSLOduration=1.011830775 podStartE2EDuration="1.011830775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:34:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:34:40.010426004 +0000 UTC m=+2340.897162341" watchObservedRunningTime="2026-07-10 23:34:40.011830775 +0000 UTC m=+2340.898566917"
Jul 10 23:34:48.354621 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:48.354587    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:34:49.339789 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:49.339748    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"
Jul 10 23:34:49.340003 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:49.339887    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"
Jul 10 23:34:49.352544 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:49.352508    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"
Jul 10 23:34:50.058223 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:34:50.058191    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"
Jul 10 23:35:39.679635 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:35:39.679598    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/ovn-acl-logging/0.log"
Jul 10 23:35:39.687010 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:35:39.686986    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/ovn-acl-logging/0.log"
Jul 10 23:35:56.421888 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:35:56.421852    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:35:56.523253 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:35:56.523221    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:36:03.885809 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:36:03.885721    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:36:49.179721 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:36:49.179678    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:36:50.103906 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:36:50.103866    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"]
Jul 10 23:36:50.130248 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:36:50.130214    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:36:54.925742 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:36:54.925703    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:37:05.127084 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.127039    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5" podUID="4b94ad69-e327-4dee-b291-b3f808132bde" containerName="main" containerID="cri-o://5e7497a1535d983e4bedb744f7de85f7b1acf49b833f46e1e5c303e071305cd7" gracePeriod=45
Jul 10 23:37:05.388190 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.388117    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"
Jul 10 23:37:05.554016 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.553986    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b94ad69-e327-4dee-b291-b3f808132bde-home\") pod \"4b94ad69-e327-4dee-b291-b3f808132bde\" (UID: \"4b94ad69-e327-4dee-b291-b3f808132bde\") "
Jul 10 23:37:05.554196 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.554053    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b94ad69-e327-4dee-b291-b3f808132bde-model-cache\") pod \"4b94ad69-e327-4dee-b291-b3f808132bde\" (UID: \"4b94ad69-e327-4dee-b291-b3f808132bde\") "
Jul 10 23:37:05.554196 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.554079    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b94ad69-e327-4dee-b291-b3f808132bde-dshm\") pod \"4b94ad69-e327-4dee-b291-b3f808132bde\" (UID: \"4b94ad69-e327-4dee-b291-b3f808132bde\") "
Jul 10 23:37:05.554196 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.554103    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b94ad69-e327-4dee-b291-b3f808132bde-tmp-dir\") pod \"4b94ad69-e327-4dee-b291-b3f808132bde\" (UID: \"4b94ad69-e327-4dee-b291-b3f808132bde\") "
Jul 10 23:37:05.554196 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.554145    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxlfq\" (UniqueName: \"kubernetes.io/projected/4b94ad69-e327-4dee-b291-b3f808132bde-kube-api-access-kxlfq\") pod \"4b94ad69-e327-4dee-b291-b3f808132bde\" (UID: \"4b94ad69-e327-4dee-b291-b3f808132bde\") "
Jul 10 23:37:05.554445 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.554210    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b94ad69-e327-4dee-b291-b3f808132bde-tls-certs\") pod \"4b94ad69-e327-4dee-b291-b3f808132bde\" (UID: \"4b94ad69-e327-4dee-b291-b3f808132bde\") "
Jul 10 23:37:05.554445 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.554316    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b94ad69-e327-4dee-b291-b3f808132bde-home" (OuterVolumeSpecName: "home") pod "4b94ad69-e327-4dee-b291-b3f808132bde" (UID: "4b94ad69-e327-4dee-b291-b3f808132bde"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:37:05.554445 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.554323    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b94ad69-e327-4dee-b291-b3f808132bde-model-cache" (OuterVolumeSpecName: "model-cache") pod "4b94ad69-e327-4dee-b291-b3f808132bde" (UID: "4b94ad69-e327-4dee-b291-b3f808132bde"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:37:05.554606 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.554450    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b94ad69-e327-4dee-b291-b3f808132bde-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4b94ad69-e327-4dee-b291-b3f808132bde" (UID: "4b94ad69-e327-4dee-b291-b3f808132bde"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:37:05.554606 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.554568    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b94ad69-e327-4dee-b291-b3f808132bde-home\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:37:05.554606 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.554588    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b94ad69-e327-4dee-b291-b3f808132bde-model-cache\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:37:05.554606 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.554604    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b94ad69-e327-4dee-b291-b3f808132bde-tmp-dir\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:37:05.556720 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.556691    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b94ad69-e327-4dee-b291-b3f808132bde-dshm" (OuterVolumeSpecName: "dshm") pod "4b94ad69-e327-4dee-b291-b3f808132bde" (UID: "4b94ad69-e327-4dee-b291-b3f808132bde"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:37:05.556853 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.556761    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b94ad69-e327-4dee-b291-b3f808132bde-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4b94ad69-e327-4dee-b291-b3f808132bde" (UID: "4b94ad69-e327-4dee-b291-b3f808132bde"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:37:05.556853 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.556777    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b94ad69-e327-4dee-b291-b3f808132bde-kube-api-access-kxlfq" (OuterVolumeSpecName: "kube-api-access-kxlfq") pod "4b94ad69-e327-4dee-b291-b3f808132bde" (UID: "4b94ad69-e327-4dee-b291-b3f808132bde"). InnerVolumeSpecName "kube-api-access-kxlfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:37:05.655698 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.655598    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b94ad69-e327-4dee-b291-b3f808132bde-dshm\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:37:05.655698 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.655630    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxlfq\" (UniqueName: \"kubernetes.io/projected/4b94ad69-e327-4dee-b291-b3f808132bde-kube-api-access-kxlfq\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:37:05.655698 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.655644    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b94ad69-e327-4dee-b291-b3f808132bde-tls-certs\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:37:05.750967 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.750928    2563 generic.go:356] "Generic (PLEG): container finished" podID="4b94ad69-e327-4dee-b291-b3f808132bde" containerID="5e7497a1535d983e4bedb744f7de85f7b1acf49b833f46e1e5c303e071305cd7" exitCode=0
Jul 10 23:37:05.751158 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.750976    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"
Jul 10 23:37:05.751158 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.751022    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5" event={"ID":"4b94ad69-e327-4dee-b291-b3f808132bde","Type":"ContainerDied","Data":"5e7497a1535d983e4bedb744f7de85f7b1acf49b833f46e1e5c303e071305cd7"}
Jul 10 23:37:05.751158 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.751069    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5" event={"ID":"4b94ad69-e327-4dee-b291-b3f808132bde","Type":"ContainerDied","Data":"11afb7995136151740e24590fe7c69de803407443c55f71614c79edea74c47e0"}
Jul 10 23:37:05.751158 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.751089    2563 scope.go:117] "RemoveContainer" containerID="5e7497a1535d983e4bedb744f7de85f7b1acf49b833f46e1e5c303e071305cd7"
Jul 10 23:37:05.760566 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.760544    2563 scope.go:117] "RemoveContainer" containerID="5e7497a1535d983e4bedb744f7de85f7b1acf49b833f46e1e5c303e071305cd7"
Jul 10 23:37:05.760901 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:37:05.760878    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e7497a1535d983e4bedb744f7de85f7b1acf49b833f46e1e5c303e071305cd7\": container with ID starting with 5e7497a1535d983e4bedb744f7de85f7b1acf49b833f46e1e5c303e071305cd7 not found: ID does not exist" containerID="5e7497a1535d983e4bedb744f7de85f7b1acf49b833f46e1e5c303e071305cd7"
Jul 10 23:37:05.761012 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.760910    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e7497a1535d983e4bedb744f7de85f7b1acf49b833f46e1e5c303e071305cd7"} err="failed to get container status \"5e7497a1535d983e4bedb744f7de85f7b1acf49b833f46e1e5c303e071305cd7\": rpc error: code = NotFound desc = could not find container \"5e7497a1535d983e4bedb744f7de85f7b1acf49b833f46e1e5c303e071305cd7\": container with ID starting with 5e7497a1535d983e4bedb744f7de85f7b1acf49b833f46e1e5c303e071305cd7 not found: ID does not exist"
Jul 10 23:37:05.766410 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.766381    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"]
Jul 10 23:37:05.768158 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:05.768134    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-27111a/auth-test-model-27111a-kserve-7448ffd8d7-n2lj5"]
Jul 10 23:37:07.556766 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:07.556720    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b94ad69-e327-4dee-b291-b3f808132bde" path="/var/lib/kubelet/pods/4b94ad69-e327-4dee-b291-b3f808132bde/volumes"
Jul 10 23:37:12.737615 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:12.737583    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:37:51.371311 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:37:51.371278    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:38:19.792869 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:19.792830    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:38:19.894295 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:19.894259    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:38:37.977591 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:37.977551    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:38:39.036132 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:39.036093    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:38:48.099765 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.099649    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"]
Jul 10 23:38:48.100222 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.100076    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b94ad69-e327-4dee-b291-b3f808132bde" containerName="main"
Jul 10 23:38:48.100222 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.100088    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b94ad69-e327-4dee-b291-b3f808132bde" containerName="main"
Jul 10 23:38:48.100222 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.100154    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b94ad69-e327-4dee-b291-b3f808132bde" containerName="main"
Jul 10 23:38:48.103461 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.103439    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"
Jul 10 23:38:48.105986 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.105952    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f5848c\"/\"default-dockercfg-2ftxv\""
Jul 10 23:38:48.106863 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.106847    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-f5848c\"/\"kube-root-ca.crt\""
Jul 10 23:38:48.106959 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.106884    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-f5848c\"/\"rate-test-model-f5848c-kserve-self-signed-certs\""
Jul 10 23:38:48.106959 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.106907    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-f5848c\"/\"openshift-service-ca.crt\""
Jul 10 23:38:48.111529 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.111501    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"]
Jul 10 23:38:48.125045 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.125017    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:38:48.238949 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.238909    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8lnp\" (UniqueName: \"kubernetes.io/projected/b09f957d-74a9-494e-9fe7-e29c9aa15baf-kube-api-access-f8lnp\") pod \"rate-test-model-f5848c-kserve-5f9f78494c-m8dvw\" (UID: \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\") " pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"
Jul 10 23:38:48.239134 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.238963    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b09f957d-74a9-494e-9fe7-e29c9aa15baf-dshm\") pod \"rate-test-model-f5848c-kserve-5f9f78494c-m8dvw\" (UID: \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\") " pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"
Jul 10 23:38:48.239134 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.238985    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b09f957d-74a9-494e-9fe7-e29c9aa15baf-model-cache\") pod \"rate-test-model-f5848c-kserve-5f9f78494c-m8dvw\" (UID: \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\") " pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"
Jul 10 23:38:48.239134 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.239002    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b09f957d-74a9-494e-9fe7-e29c9aa15baf-tls-certs\") pod \"rate-test-model-f5848c-kserve-5f9f78494c-m8dvw\" (UID: \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\") " pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"
Jul 10 23:38:48.239134 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.239024    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b09f957d-74a9-494e-9fe7-e29c9aa15baf-home\") pod \"rate-test-model-f5848c-kserve-5f9f78494c-m8dvw\" (UID: \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\") " pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"
Jul 10 23:38:48.239134 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.239096    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b09f957d-74a9-494e-9fe7-e29c9aa15baf-tmp-dir\") pod \"rate-test-model-f5848c-kserve-5f9f78494c-m8dvw\" (UID: \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\") " pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"
Jul 10 23:38:48.340125 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.340083    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b09f957d-74a9-494e-9fe7-e29c9aa15baf-tmp-dir\") pod \"rate-test-model-f5848c-kserve-5f9f78494c-m8dvw\" (UID: \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\") " pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"
Jul 10 23:38:48.340331 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.340159    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8lnp\" (UniqueName: \"kubernetes.io/projected/b09f957d-74a9-494e-9fe7-e29c9aa15baf-kube-api-access-f8lnp\") pod \"rate-test-model-f5848c-kserve-5f9f78494c-m8dvw\" (UID: \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\") " pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"
Jul 10 23:38:48.340331 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.340198    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b09f957d-74a9-494e-9fe7-e29c9aa15baf-dshm\") pod \"rate-test-model-f5848c-kserve-5f9f78494c-m8dvw\" (UID: \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\") " pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"
Jul 10 23:38:48.340331 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.340224    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b09f957d-74a9-494e-9fe7-e29c9aa15baf-model-cache\") pod \"rate-test-model-f5848c-kserve-5f9f78494c-m8dvw\" (UID: \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\") " pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"
Jul 10 23:38:48.340331 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.340249    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b09f957d-74a9-494e-9fe7-e29c9aa15baf-tls-certs\") pod \"rate-test-model-f5848c-kserve-5f9f78494c-m8dvw\" (UID: \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\") " pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"
Jul 10 23:38:48.340601 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.340517    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b09f957d-74a9-494e-9fe7-e29c9aa15baf-tmp-dir\") pod \"rate-test-model-f5848c-kserve-5f9f78494c-m8dvw\" (UID: \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\") " pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"
Jul 10 23:38:48.340601 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.340584    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b09f957d-74a9-494e-9fe7-e29c9aa15baf-model-cache\") pod \"rate-test-model-f5848c-kserve-5f9f78494c-m8dvw\" (UID: \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\") " pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"
Jul 10 23:38:48.340710 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.340641    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b09f957d-74a9-494e-9fe7-e29c9aa15baf-home\") pod \"rate-test-model-f5848c-kserve-5f9f78494c-m8dvw\" (UID: \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\") " pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"
Jul 10 23:38:48.341114 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.341080    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b09f957d-74a9-494e-9fe7-e29c9aa15baf-home\") pod \"rate-test-model-f5848c-kserve-5f9f78494c-m8dvw\" (UID: \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\") " pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"
Jul 10 23:38:48.342803 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.342776    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b09f957d-74a9-494e-9fe7-e29c9aa15baf-dshm\") pod \"rate-test-model-f5848c-kserve-5f9f78494c-m8dvw\" (UID: \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\") " pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"
Jul 10 23:38:48.342916 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.342891    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b09f957d-74a9-494e-9fe7-e29c9aa15baf-tls-certs\") pod \"rate-test-model-f5848c-kserve-5f9f78494c-m8dvw\" (UID: \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\") " pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"
Jul 10 23:38:48.348115 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.348090    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8lnp\" (UniqueName: \"kubernetes.io/projected/b09f957d-74a9-494e-9fe7-e29c9aa15baf-kube-api-access-f8lnp\") pod \"rate-test-model-f5848c-kserve-5f9f78494c-m8dvw\" (UID: \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\") " pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"
Jul 10 23:38:48.413672 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.413569    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"
Jul 10 23:38:48.556151 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:48.556122    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"]
Jul 10 23:38:48.558562 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:38:48.558537    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb09f957d_74a9_494e_9fe7_e29c9aa15baf.slice/crio-3beed6818fc0d453d399a0fd8bf0b1d5aad3fdcc2d387a8a1a3e7ff44c79118d WatchSource:0}: Error finding container 3beed6818fc0d453d399a0fd8bf0b1d5aad3fdcc2d387a8a1a3e7ff44c79118d: Status 404 returned error can't find the container with id 3beed6818fc0d453d399a0fd8bf0b1d5aad3fdcc2d387a8a1a3e7ff44c79118d
Jul 10 23:38:49.298003 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:49.297947    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw" event={"ID":"b09f957d-74a9-494e-9fe7-e29c9aa15baf","Type":"ContainerStarted","Data":"ef394a772e91228a2ec193d6e9a9d085bf42c22ea3dc31ae9dae7daffcb53787"}
Jul 10 23:38:49.298003 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:49.298003    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw" event={"ID":"b09f957d-74a9-494e-9fe7-e29c9aa15baf","Type":"ContainerStarted","Data":"3beed6818fc0d453d399a0fd8bf0b1d5aad3fdcc2d387a8a1a3e7ff44c79118d"}
Jul 10 23:38:49.314482 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:49.314399    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw" podStartSLOduration=1.31435503 podStartE2EDuration="1.31435503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:38:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:38:49.313063801 +0000 UTC m=+2590.199799946" watchObservedRunningTime="2026-07-10 23:38:49.31435503 +0000 UTC m=+2590.201091172"
Jul 10 23:38:58.414438 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:58.414388    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"
Jul 10 23:38:58.414947 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:58.414451    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"
Jul 10 23:38:58.426979 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:58.426949    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"
Jul 10 23:38:59.361242 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:38:59.361210    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"
Jul 10 23:39:01.250373 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:39:01.250280    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:39:06.878544 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:39:06.878502    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:39:06.978800 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:39:06.978765    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:39:14.658303 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:39:14.658258    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:39:14.758444 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:39:14.758411    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:39:55.343276 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:39:55.343234    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"]
Jul 10 23:39:55.365294 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:39:55.365262    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:39:56.424614 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:39:56.424578    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:40:10.367584 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:10.367540    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw" podUID="b09f957d-74a9-494e-9fe7-e29c9aa15baf" containerName="main" containerID="cri-o://ef394a772e91228a2ec193d6e9a9d085bf42c22ea3dc31ae9dae7daffcb53787" gracePeriod=45
Jul 10 23:40:10.632113 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:10.632047    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"
Jul 10 23:40:10.722482 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:10.722448    2563 generic.go:356] "Generic (PLEG): container finished" podID="b09f957d-74a9-494e-9fe7-e29c9aa15baf" containerID="ef394a772e91228a2ec193d6e9a9d085bf42c22ea3dc31ae9dae7daffcb53787" exitCode=0
Jul 10 23:40:10.722660 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:10.722496    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"
Jul 10 23:40:10.722660 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:10.722516    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw" event={"ID":"b09f957d-74a9-494e-9fe7-e29c9aa15baf","Type":"ContainerDied","Data":"ef394a772e91228a2ec193d6e9a9d085bf42c22ea3dc31ae9dae7daffcb53787"}
Jul 10 23:40:10.722660 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:10.722548    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw" event={"ID":"b09f957d-74a9-494e-9fe7-e29c9aa15baf","Type":"ContainerDied","Data":"3beed6818fc0d453d399a0fd8bf0b1d5aad3fdcc2d387a8a1a3e7ff44c79118d"}
Jul 10 23:40:10.722660 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:10.722565    2563 scope.go:117] "RemoveContainer" containerID="ef394a772e91228a2ec193d6e9a9d085bf42c22ea3dc31ae9dae7daffcb53787"
Jul 10 23:40:10.731394 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:10.731346    2563 scope.go:117] "RemoveContainer" containerID="ef394a772e91228a2ec193d6e9a9d085bf42c22ea3dc31ae9dae7daffcb53787"
Jul 10 23:40:10.731711 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:40:10.731689    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef394a772e91228a2ec193d6e9a9d085bf42c22ea3dc31ae9dae7daffcb53787\": container with ID starting with ef394a772e91228a2ec193d6e9a9d085bf42c22ea3dc31ae9dae7daffcb53787 not found: ID does not exist" containerID="ef394a772e91228a2ec193d6e9a9d085bf42c22ea3dc31ae9dae7daffcb53787"
Jul 10 23:40:10.731787 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:10.731728    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef394a772e91228a2ec193d6e9a9d085bf42c22ea3dc31ae9dae7daffcb53787"} err="failed to get container status \"ef394a772e91228a2ec193d6e9a9d085bf42c22ea3dc31ae9dae7daffcb53787\": rpc error: code = NotFound desc = could not find container \"ef394a772e91228a2ec193d6e9a9d085bf42c22ea3dc31ae9dae7daffcb53787\": container with ID starting with ef394a772e91228a2ec193d6e9a9d085bf42c22ea3dc31ae9dae7daffcb53787 not found: ID does not exist"
Jul 10 23:40:10.749115 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:10.749079    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8lnp\" (UniqueName: \"kubernetes.io/projected/b09f957d-74a9-494e-9fe7-e29c9aa15baf-kube-api-access-f8lnp\") pod \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\" (UID: \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\") "
Jul 10 23:40:10.749262 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:10.749130    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b09f957d-74a9-494e-9fe7-e29c9aa15baf-tmp-dir\") pod \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\" (UID: \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\") "
Jul 10 23:40:10.749262 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:10.749157    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b09f957d-74a9-494e-9fe7-e29c9aa15baf-model-cache\") pod \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\" (UID: \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\") "
Jul 10 23:40:10.749262 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:10.749185    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b09f957d-74a9-494e-9fe7-e29c9aa15baf-tls-certs\") pod \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\" (UID: \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\") "
Jul 10 23:40:10.749459 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:10.749288    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b09f957d-74a9-494e-9fe7-e29c9aa15baf-dshm\") pod \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\" (UID: \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\") "
Jul 10 23:40:10.749459 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:10.749328    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b09f957d-74a9-494e-9fe7-e29c9aa15baf-home\") pod \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\" (UID: \"b09f957d-74a9-494e-9fe7-e29c9aa15baf\") "
Jul 10 23:40:10.749562 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:10.749472    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b09f957d-74a9-494e-9fe7-e29c9aa15baf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b09f957d-74a9-494e-9fe7-e29c9aa15baf" (UID: "b09f957d-74a9-494e-9fe7-e29c9aa15baf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:40:10.749562 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:10.749493    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b09f957d-74a9-494e-9fe7-e29c9aa15baf-model-cache" (OuterVolumeSpecName: "model-cache") pod "b09f957d-74a9-494e-9fe7-e29c9aa15baf" (UID: "b09f957d-74a9-494e-9fe7-e29c9aa15baf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:40:10.749655 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:10.749624    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b09f957d-74a9-494e-9fe7-e29c9aa15baf-home" (OuterVolumeSpecName: "home") pod "b09f957d-74a9-494e-9fe7-e29c9aa15baf" (UID: "b09f957d-74a9-494e-9fe7-e29c9aa15baf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:40:10.749774 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:10.749755    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b09f957d-74a9-494e-9fe7-e29c9aa15baf-tmp-dir\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:40:10.749836 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:10.749777    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b09f957d-74a9-494e-9fe7-e29c9aa15baf-model-cache\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:40:10.749836 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:10.749784    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b09f957d-74a9-494e-9fe7-e29c9aa15baf-home\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:40:10.751644 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:10.751617    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b09f957d-74a9-494e-9fe7-e29c9aa15baf-dshm" (OuterVolumeSpecName: "dshm") pod "b09f957d-74a9-494e-9fe7-e29c9aa15baf" (UID: "b09f957d-74a9-494e-9fe7-e29c9aa15baf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:40:10.751754 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:10.751679    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b09f957d-74a9-494e-9fe7-e29c9aa15baf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b09f957d-74a9-494e-9fe7-e29c9aa15baf" (UID: "b09f957d-74a9-494e-9fe7-e29c9aa15baf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:40:10.751754 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:10.751694    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b09f957d-74a9-494e-9fe7-e29c9aa15baf-kube-api-access-f8lnp" (OuterVolumeSpecName: "kube-api-access-f8lnp") pod "b09f957d-74a9-494e-9fe7-e29c9aa15baf" (UID: "b09f957d-74a9-494e-9fe7-e29c9aa15baf"). InnerVolumeSpecName "kube-api-access-f8lnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:40:10.850878 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:10.850843    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8lnp\" (UniqueName: \"kubernetes.io/projected/b09f957d-74a9-494e-9fe7-e29c9aa15baf-kube-api-access-f8lnp\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:40:10.850878 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:10.850872    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b09f957d-74a9-494e-9fe7-e29c9aa15baf-tls-certs\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:40:10.850878 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:10.850883    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b09f957d-74a9-494e-9fe7-e29c9aa15baf-dshm\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:40:11.044097 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:11.044056    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"]
Jul 10 23:40:11.045872 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:11.045842    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-f5848c/rate-test-model-f5848c-kserve-5f9f78494c-m8dvw"]
Jul 10 23:40:11.556968 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:11.556934    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b09f957d-74a9-494e-9fe7-e29c9aa15baf" path="/var/lib/kubelet/pods/b09f957d-74a9-494e-9fe7-e29c9aa15baf/volumes"
Jul 10 23:40:15.326080 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:15.326045    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:40:15.752307 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:15.752270    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:40:15.854335 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:15.854299    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:40:16.014114 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:16.014028    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:40:21.506527 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:21.506480    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:40:21.606623 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:21.606582    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:40:21.753989 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:21.753943    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"]
Jul 10 23:40:21.754264 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:21.754238    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp" podUID="906d9f75-9dfe-4014-b699-22e47c5c0c40" containerName="istio-proxy" containerID="cri-o://a1365c51c163fc0610414033b85ddf3bed5fe1cf9bcdd86a467ff911c8fce412" gracePeriod=30
Jul 10 23:40:21.766177 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:21.766096    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:40:22.013544 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:22.013513    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:40:22.176900 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:22.176800    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:40:22.591415 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:22.591378    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:40:22.728060 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:22.728022    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:40:22.830480 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:22.830439    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:40:22.954648 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:22.954561    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-df5db5888-sbmq9"]
Jul 10 23:40:22.954899 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:22.954842    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-df5db5888-sbmq9" podUID="3e7ab61b-0379-445c-8080-538f710aedaf" containerName="maas-api" containerID="cri-o://1edf4edd4603765b8bb94a7ba80e51eea50012a5af57aebdce44c0a00cc786f1" gracePeriod=30
Jul 10 23:40:22.984794 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:22.984755    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:40:23.406961 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:23.406931    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-df5db5888-sbmq9"
Jul 10 23:40:23.470331 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:23.470293    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e7ab61b-0379-445c-8080-538f710aedaf-maas-api-tls\") pod \"3e7ab61b-0379-445c-8080-538f710aedaf\" (UID: \"3e7ab61b-0379-445c-8080-538f710aedaf\") "
Jul 10 23:40:23.470548 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:23.470344    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nc22b\" (UniqueName: \"kubernetes.io/projected/3e7ab61b-0379-445c-8080-538f710aedaf-kube-api-access-nc22b\") pod \"3e7ab61b-0379-445c-8080-538f710aedaf\" (UID: \"3e7ab61b-0379-445c-8080-538f710aedaf\") "
Jul 10 23:40:23.470548 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:23.470434    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e7ab61b-0379-445c-8080-538f710aedaf-openshift-service-ca\") pod \"3e7ab61b-0379-445c-8080-538f710aedaf\" (UID: \"3e7ab61b-0379-445c-8080-538f710aedaf\") "
Jul 10 23:40:23.470889 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:23.470863    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e7ab61b-0379-445c-8080-538f710aedaf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3e7ab61b-0379-445c-8080-538f710aedaf" (UID: "3e7ab61b-0379-445c-8080-538f710aedaf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:40:23.472783 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:23.472748    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e7ab61b-0379-445c-8080-538f710aedaf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3e7ab61b-0379-445c-8080-538f710aedaf" (UID: "3e7ab61b-0379-445c-8080-538f710aedaf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 23:40:23.472894 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:23.472755    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e7ab61b-0379-445c-8080-538f710aedaf-kube-api-access-nc22b" (OuterVolumeSpecName: "kube-api-access-nc22b") pod "3e7ab61b-0379-445c-8080-538f710aedaf" (UID: "3e7ab61b-0379-445c-8080-538f710aedaf"). InnerVolumeSpecName "kube-api-access-nc22b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:40:23.578216 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:23.572741    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e7ab61b-0379-445c-8080-538f710aedaf-openshift-service-ca\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:40:23.578216 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:23.572787    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e7ab61b-0379-445c-8080-538f710aedaf-maas-api-tls\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:40:23.578216 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:23.572802    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nc22b\" (UniqueName: \"kubernetes.io/projected/3e7ab61b-0379-445c-8080-538f710aedaf-kube-api-access-nc22b\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:40:23.796273 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:23.796228    2563 generic.go:356] "Generic (PLEG): container finished" podID="3e7ab61b-0379-445c-8080-538f710aedaf" containerID="1edf4edd4603765b8bb94a7ba80e51eea50012a5af57aebdce44c0a00cc786f1" exitCode=0
Jul 10 23:40:23.796714 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:23.796300    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-df5db5888-sbmq9"
Jul 10 23:40:23.796714 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:23.796303    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-df5db5888-sbmq9" event={"ID":"3e7ab61b-0379-445c-8080-538f710aedaf","Type":"ContainerDied","Data":"1edf4edd4603765b8bb94a7ba80e51eea50012a5af57aebdce44c0a00cc786f1"}
Jul 10 23:40:23.796714 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:23.796461    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-df5db5888-sbmq9" event={"ID":"3e7ab61b-0379-445c-8080-538f710aedaf","Type":"ContainerDied","Data":"b2f295af3640b8a214ab7feb15d10a90bba0d3456d3eacf6525b475af589fe5d"}
Jul 10 23:40:23.796714 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:23.796484    2563 scope.go:117] "RemoveContainer" containerID="1edf4edd4603765b8bb94a7ba80e51eea50012a5af57aebdce44c0a00cc786f1"
Jul 10 23:40:23.805613 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:23.805589    2563 scope.go:117] "RemoveContainer" containerID="1edf4edd4603765b8bb94a7ba80e51eea50012a5af57aebdce44c0a00cc786f1"
Jul 10 23:40:23.805922 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:40:23.805902    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1edf4edd4603765b8bb94a7ba80e51eea50012a5af57aebdce44c0a00cc786f1\": container with ID starting with 1edf4edd4603765b8bb94a7ba80e51eea50012a5af57aebdce44c0a00cc786f1 not found: ID does not exist" containerID="1edf4edd4603765b8bb94a7ba80e51eea50012a5af57aebdce44c0a00cc786f1"
Jul 10 23:40:23.805967 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:23.805935    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1edf4edd4603765b8bb94a7ba80e51eea50012a5af57aebdce44c0a00cc786f1"} err="failed to get container status \"1edf4edd4603765b8bb94a7ba80e51eea50012a5af57aebdce44c0a00cc786f1\": rpc error: code = NotFound desc = could not find container \"1edf4edd4603765b8bb94a7ba80e51eea50012a5af57aebdce44c0a00cc786f1\": container with ID starting with 1edf4edd4603765b8bb94a7ba80e51eea50012a5af57aebdce44c0a00cc786f1 not found: ID does not exist"
Jul 10 23:40:23.811326 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:23.811296    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-df5db5888-sbmq9"]
Jul 10 23:40:23.815417 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:23.815326    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-27111a-df5db5888-sbmq9"]
Jul 10 23:40:25.556065 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:25.556032    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e7ab61b-0379-445c-8080-538f710aedaf" path="/var/lib/kubelet/pods/3e7ab61b-0379-445c-8080-538f710aedaf/volumes"
Jul 10 23:40:27.013089 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.013060    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:40:27.105433 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.105333    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-workload-socket\") pod \"906d9f75-9dfe-4014-b699-22e47c5c0c40\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") "
Jul 10 23:40:27.105433 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.105413    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fj6jq\" (UniqueName: \"kubernetes.io/projected/906d9f75-9dfe-4014-b699-22e47c5c0c40-kube-api-access-fj6jq\") pod \"906d9f75-9dfe-4014-b699-22e47c5c0c40\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") "
Jul 10 23:40:27.105679 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.105442    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-credential-socket\") pod \"906d9f75-9dfe-4014-b699-22e47c5c0c40\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") "
Jul 10 23:40:27.105679 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.105474    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/906d9f75-9dfe-4014-b699-22e47c5c0c40-istio-podinfo\") pod \"906d9f75-9dfe-4014-b699-22e47c5c0c40\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") "
Jul 10 23:40:27.105679 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.105494    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/906d9f75-9dfe-4014-b699-22e47c5c0c40-istiod-ca-cert\") pod \"906d9f75-9dfe-4014-b699-22e47c5c0c40\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") "
Jul 10 23:40:27.105679 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.105600    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "906d9f75-9dfe-4014-b699-22e47c5c0c40" (UID: "906d9f75-9dfe-4014-b699-22e47c5c0c40"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:40:27.105679 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.105640    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-istio-envoy\") pod \"906d9f75-9dfe-4014-b699-22e47c5c0c40\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") "
Jul 10 23:40:27.105939 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.105685    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-istio-data\") pod \"906d9f75-9dfe-4014-b699-22e47c5c0c40\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") "
Jul 10 23:40:27.105939 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.105720    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-workload-certs\") pod \"906d9f75-9dfe-4014-b699-22e47c5c0c40\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") "
Jul 10 23:40:27.105939 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.105779    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/906d9f75-9dfe-4014-b699-22e47c5c0c40-istio-token\") pod \"906d9f75-9dfe-4014-b699-22e47c5c0c40\" (UID: \"906d9f75-9dfe-4014-b699-22e47c5c0c40\") "
Jul 10 23:40:27.105939 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.105792    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "906d9f75-9dfe-4014-b699-22e47c5c0c40" (UID: "906d9f75-9dfe-4014-b699-22e47c5c0c40"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:40:27.106131 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.105979    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/906d9f75-9dfe-4014-b699-22e47c5c0c40-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "906d9f75-9dfe-4014-b699-22e47c5c0c40" (UID: "906d9f75-9dfe-4014-b699-22e47c5c0c40"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:40:27.106131 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.105991    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "906d9f75-9dfe-4014-b699-22e47c5c0c40" (UID: "906d9f75-9dfe-4014-b699-22e47c5c0c40"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:40:27.106131 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.106118    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-workload-socket\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:40:27.106699 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.106137    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-credential-socket\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:40:27.106699 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.106151    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/906d9f75-9dfe-4014-b699-22e47c5c0c40-istiod-ca-cert\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:40:27.106699 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.106165    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-workload-certs\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:40:27.106699 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.106423    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-istio-data" (OuterVolumeSpecName: "istio-data") pod "906d9f75-9dfe-4014-b699-22e47c5c0c40" (UID: "906d9f75-9dfe-4014-b699-22e47c5c0c40"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:40:27.107972 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.107939    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "906d9f75-9dfe-4014-b699-22e47c5c0c40" (UID: "906d9f75-9dfe-4014-b699-22e47c5c0c40"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:40:27.108062 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.107977    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/906d9f75-9dfe-4014-b699-22e47c5c0c40-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "906d9f75-9dfe-4014-b699-22e47c5c0c40" (UID: "906d9f75-9dfe-4014-b699-22e47c5c0c40"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 23:40:27.108062 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.107976    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/906d9f75-9dfe-4014-b699-22e47c5c0c40-kube-api-access-fj6jq" (OuterVolumeSpecName: "kube-api-access-fj6jq") pod "906d9f75-9dfe-4014-b699-22e47c5c0c40" (UID: "906d9f75-9dfe-4014-b699-22e47c5c0c40"). InnerVolumeSpecName "kube-api-access-fj6jq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:40:27.108230 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.108207    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/906d9f75-9dfe-4014-b699-22e47c5c0c40-istio-token" (OuterVolumeSpecName: "istio-token") pod "906d9f75-9dfe-4014-b699-22e47c5c0c40" (UID: "906d9f75-9dfe-4014-b699-22e47c5c0c40"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:40:27.207244 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.207189    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/906d9f75-9dfe-4014-b699-22e47c5c0c40-istio-podinfo\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:40:27.207244 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.207235    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-istio-envoy\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:40:27.207244 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.207244    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/906d9f75-9dfe-4014-b699-22e47c5c0c40-istio-data\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:40:27.207244 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.207252    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/906d9f75-9dfe-4014-b699-22e47c5c0c40-istio-token\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:40:27.207244 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.207260    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fj6jq\" (UniqueName: \"kubernetes.io/projected/906d9f75-9dfe-4014-b699-22e47c5c0c40-kube-api-access-fj6jq\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:40:27.818862 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.818770    2563 generic.go:356] "Generic (PLEG): container finished" podID="906d9f75-9dfe-4014-b699-22e47c5c0c40" containerID="a1365c51c163fc0610414033b85ddf3bed5fe1cf9bcdd86a467ff911c8fce412" exitCode=0
Jul 10 23:40:27.818862 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.818818    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"
Jul 10 23:40:27.819097 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.818854    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp" event={"ID":"906d9f75-9dfe-4014-b699-22e47c5c0c40","Type":"ContainerDied","Data":"a1365c51c163fc0610414033b85ddf3bed5fe1cf9bcdd86a467ff911c8fce412"}
Jul 10 23:40:27.819097 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.818909    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp" event={"ID":"906d9f75-9dfe-4014-b699-22e47c5c0c40","Type":"ContainerDied","Data":"22eda82cb5c57a7157e65c9c213b7dec45d2485418b253c27d51193edcdd065f"}
Jul 10 23:40:27.819097 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.818933    2563 scope.go:117] "RemoveContainer" containerID="a1365c51c163fc0610414033b85ddf3bed5fe1cf9bcdd86a467ff911c8fce412"
Jul 10 23:40:27.828090 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.828061    2563 scope.go:117] "RemoveContainer" containerID="430deb35447f03394fd35ae35815fa70ef9034c489a2b1061fe0a72f5c768125"
Jul 10 23:40:27.835392 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.835330    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"]
Jul 10 23:40:27.837487 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.837468    2563 scope.go:117] "RemoveContainer" containerID="a1365c51c163fc0610414033b85ddf3bed5fe1cf9bcdd86a467ff911c8fce412"
Jul 10 23:40:27.837792 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:40:27.837771    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1365c51c163fc0610414033b85ddf3bed5fe1cf9bcdd86a467ff911c8fce412\": container with ID starting with a1365c51c163fc0610414033b85ddf3bed5fe1cf9bcdd86a467ff911c8fce412 not found: ID does not exist" containerID="a1365c51c163fc0610414033b85ddf3bed5fe1cf9bcdd86a467ff911c8fce412"
Jul 10 23:40:27.837877 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.837800    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1365c51c163fc0610414033b85ddf3bed5fe1cf9bcdd86a467ff911c8fce412"} err="failed to get container status \"a1365c51c163fc0610414033b85ddf3bed5fe1cf9bcdd86a467ff911c8fce412\": rpc error: code = NotFound desc = could not find container \"a1365c51c163fc0610414033b85ddf3bed5fe1cf9bcdd86a467ff911c8fce412\": container with ID starting with a1365c51c163fc0610414033b85ddf3bed5fe1cf9bcdd86a467ff911c8fce412 not found: ID does not exist"
Jul 10 23:40:27.837877 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.837819    2563 scope.go:117] "RemoveContainer" containerID="430deb35447f03394fd35ae35815fa70ef9034c489a2b1061fe0a72f5c768125"
Jul 10 23:40:27.838087 ip-10-0-141-239 kubenswrapper[2563]: E0710 23:40:27.838068    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"430deb35447f03394fd35ae35815fa70ef9034c489a2b1061fe0a72f5c768125\": container with ID starting with 430deb35447f03394fd35ae35815fa70ef9034c489a2b1061fe0a72f5c768125 not found: ID does not exist" containerID="430deb35447f03394fd35ae35815fa70ef9034c489a2b1061fe0a72f5c768125"
Jul 10 23:40:27.838126 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.838094    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"430deb35447f03394fd35ae35815fa70ef9034c489a2b1061fe0a72f5c768125"} err="failed to get container status \"430deb35447f03394fd35ae35815fa70ef9034c489a2b1061fe0a72f5c768125\": rpc error: code = NotFound desc = could not find container \"430deb35447f03394fd35ae35815fa70ef9034c489a2b1061fe0a72f5c768125\": container with ID starting with 430deb35447f03394fd35ae35815fa70ef9034c489a2b1061fe0a72f5c768125 not found: ID does not exist"
Jul 10 23:40:27.840909 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:27.840881    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-f5848c-openshift-default-56f8898bd7-7mkwp"]
Jul 10 23:40:28.365851 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:28.365822    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:40:28.421637 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:28.421601    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:40:28.606200 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:28.606162    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"]
Jul 10 23:40:28.606430 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:28.606403    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw" podUID="10c229cb-0d97-4fc0-af69-032d293f26af" containerName="istio-proxy" containerID="cri-o://37c716a1bc583d9099b8cfceef72b7f5dcaaa42b399b68dfd92491c207b7f805" gracePeriod=30
Jul 10 23:40:28.620927 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:28.620830    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:40:28.904277 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:28.904184    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:40:29.129264 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:29.129225    2563 patch_prober.go:28] interesting pod/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 10 23:40:29.129431 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:29.129289    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw" podUID="10c229cb-0d97-4fc0-af69-032d293f26af" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 23:40:29.281057 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:29.281015    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-684dfc7c65-gssr4_78f28a98-48be-4320-ace0-1b5297b6fbe0/authorino/0.log"
Jul 10 23:40:29.558406 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:29.558293    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="906d9f75-9dfe-4014-b699-22e47c5c0c40" path="/var/lib/kubelet/pods/906d9f75-9dfe-4014-b699-22e47c5c0c40/volumes"
Jul 10 23:40:33.858128 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:33.858094    2563 generic.go:356] "Generic (PLEG): container finished" podID="10c229cb-0d97-4fc0-af69-032d293f26af" containerID="37c716a1bc583d9099b8cfceef72b7f5dcaaa42b399b68dfd92491c207b7f805" exitCode=0
Jul 10 23:40:33.858535 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:33.858169    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw" event={"ID":"10c229cb-0d97-4fc0-af69-032d293f26af","Type":"ContainerDied","Data":"37c716a1bc583d9099b8cfceef72b7f5dcaaa42b399b68dfd92491c207b7f805"}
Jul 10 23:40:33.858535 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:33.858232    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw" event={"ID":"10c229cb-0d97-4fc0-af69-032d293f26af","Type":"ContainerDied","Data":"131ebc1eff29c6f6c9c44f1089491ac530c11a8bf98a8a15e84165016ed80670"}
Jul 10 23:40:33.858535 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:33.858250    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="131ebc1eff29c6f6c9c44f1089491ac530c11a8bf98a8a15e84165016ed80670"
Jul 10 23:40:33.858535 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:33.858264    2563 scope.go:117] "RemoveContainer" containerID="18c22abc8d3b260cec80a80c4999126e0988d2188f32b1154aacd35c6828e3e8"
Jul 10 23:40:33.874710 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:33.874686    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:40:33.971911 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:33.971874    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-istio-envoy\") pod \"10c229cb-0d97-4fc0-af69-032d293f26af\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") "
Jul 10 23:40:33.971911 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:33.971914    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-istio-data\") pod \"10c229cb-0d97-4fc0-af69-032d293f26af\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") "
Jul 10 23:40:33.972159 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:33.971945    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-credential-socket\") pod \"10c229cb-0d97-4fc0-af69-032d293f26af\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") "
Jul 10 23:40:33.972159 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:33.971989    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-workload-socket\") pod \"10c229cb-0d97-4fc0-af69-032d293f26af\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") "
Jul 10 23:40:33.972159 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:33.972020    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10c229cb-0d97-4fc0-af69-032d293f26af-istiod-ca-cert\") pod \"10c229cb-0d97-4fc0-af69-032d293f26af\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") "
Jul 10 23:40:33.972159 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:33.972046    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j55sp\" (UniqueName: \"kubernetes.io/projected/10c229cb-0d97-4fc0-af69-032d293f26af-kube-api-access-j55sp\") pod \"10c229cb-0d97-4fc0-af69-032d293f26af\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") "
Jul 10 23:40:33.972159 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:33.972085    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10c229cb-0d97-4fc0-af69-032d293f26af-istio-podinfo\") pod \"10c229cb-0d97-4fc0-af69-032d293f26af\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") "
Jul 10 23:40:33.972159 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:33.972119    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10c229cb-0d97-4fc0-af69-032d293f26af-istio-token\") pod \"10c229cb-0d97-4fc0-af69-032d293f26af\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") "
Jul 10 23:40:33.972159 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:33.972157    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-workload-certs\") pod \"10c229cb-0d97-4fc0-af69-032d293f26af\" (UID: \"10c229cb-0d97-4fc0-af69-032d293f26af\") "
Jul 10 23:40:33.972509 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:33.972354    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "10c229cb-0d97-4fc0-af69-032d293f26af" (UID: "10c229cb-0d97-4fc0-af69-032d293f26af"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:40:33.972509 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:33.972399    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "10c229cb-0d97-4fc0-af69-032d293f26af" (UID: "10c229cb-0d97-4fc0-af69-032d293f26af"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:40:33.972622 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:33.972554    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-credential-socket\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:40:33.972622 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:33.972571    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-workload-socket\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:40:33.972622 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:33.972596    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "10c229cb-0d97-4fc0-af69-032d293f26af" (UID: "10c229cb-0d97-4fc0-af69-032d293f26af"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:40:33.972811 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:33.972689    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10c229cb-0d97-4fc0-af69-032d293f26af-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "10c229cb-0d97-4fc0-af69-032d293f26af" (UID: "10c229cb-0d97-4fc0-af69-032d293f26af"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 23:40:33.972879 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:33.972846    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-istio-data" (OuterVolumeSpecName: "istio-data") pod "10c229cb-0d97-4fc0-af69-032d293f26af" (UID: "10c229cb-0d97-4fc0-af69-032d293f26af"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:40:33.974316 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:33.974288    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "10c229cb-0d97-4fc0-af69-032d293f26af" (UID: "10c229cb-0d97-4fc0-af69-032d293f26af"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 23:40:33.974521 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:33.974500    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/10c229cb-0d97-4fc0-af69-032d293f26af-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "10c229cb-0d97-4fc0-af69-032d293f26af" (UID: "10c229cb-0d97-4fc0-af69-032d293f26af"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 23:40:33.974597 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:33.974534    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10c229cb-0d97-4fc0-af69-032d293f26af-kube-api-access-j55sp" (OuterVolumeSpecName: "kube-api-access-j55sp") pod "10c229cb-0d97-4fc0-af69-032d293f26af" (UID: "10c229cb-0d97-4fc0-af69-032d293f26af"). InnerVolumeSpecName "kube-api-access-j55sp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:40:33.974704 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:33.974688    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10c229cb-0d97-4fc0-af69-032d293f26af-istio-token" (OuterVolumeSpecName: "istio-token") pod "10c229cb-0d97-4fc0-af69-032d293f26af" (UID: "10c229cb-0d97-4fc0-af69-032d293f26af"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 23:40:34.073721 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:34.073678    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-workload-certs\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:40:34.073721 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:34.073714    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-istio-envoy\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:40:34.073721 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:34.073722    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10c229cb-0d97-4fc0-af69-032d293f26af-istio-data\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:40:34.073721 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:34.073730    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10c229cb-0d97-4fc0-af69-032d293f26af-istiod-ca-cert\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:40:34.073995 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:34.073738    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j55sp\" (UniqueName: \"kubernetes.io/projected/10c229cb-0d97-4fc0-af69-032d293f26af-kube-api-access-j55sp\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:40:34.073995 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:34.073747    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10c229cb-0d97-4fc0-af69-032d293f26af-istio-podinfo\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:40:34.073995 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:34.073757    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10c229cb-0d97-4fc0-af69-032d293f26af-istio-token\") on node \"ip-10-0-141-239.ec2.internal\" DevicePath \"\""
Jul 10 23:40:34.864700 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:34.864668    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"
Jul 10 23:40:34.900118 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:34.900081    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"]
Jul 10 23:40:34.903273 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:34.903241    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-27111a-openshift-default-547b56db9f-5vwkw"]
Jul 10 23:40:35.037901 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:35.037869    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58bf55498c-lxs9p_96854ace-3322-4255-94bd-eb446df0a254/copy-manifests/0.log"
Jul 10 23:40:35.044723 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:35.044700    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58bf55498c-lxs9p_96854ace-3322-4255-94bd-eb446df0a254/manager/0.log"
Jul 10 23:40:35.562643 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:35.562603    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10c229cb-0d97-4fc0-af69-032d293f26af" path="/var/lib/kubelet/pods/10c229cb-0d97-4fc0-af69-032d293f26af/volumes"
Jul 10 23:40:37.032270 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:37.032239    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-684dfc7c65-gssr4_78f28a98-48be-4320-ace0-1b5297b6fbe0/authorino/0.log"
Jul 10 23:40:37.929749 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:37.929715    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-xr96m_09245303-0104-4b38-86ed-caa866cbf92d/manager/0.log"
Jul 10 23:40:38.665355 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:38.665326    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7947c57c45-c46r7_98989cf4-283f-4a9d-82b0-9b4f967d39c9/kube-auth-proxy/0.log"
Jul 10 23:40:39.540643 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:39.540617    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-btjbz_943a2f00-01e2-4006-aefb-3c1c0c6849a8/main/0.log"
Jul 10 23:40:39.713169 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:39.713128    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/ovn-acl-logging/0.log"
Jul 10 23:40:39.722352 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:39.722326    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/ovn-acl-logging/0.log"
Jul 10 23:40:39.815750 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:39.815655    2563 scope.go:117] "RemoveContainer" containerID="37c716a1bc583d9099b8cfceef72b7f5dcaaa42b399b68dfd92491c207b7f805"
Jul 10 23:40:44.293597 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.293555    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pjdll/must-gather-lql29"]
Jul 10 23:40:44.293971 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.293908    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10c229cb-0d97-4fc0-af69-032d293f26af" containerName="istio-proxy"
Jul 10 23:40:44.293971 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.293919    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="10c229cb-0d97-4fc0-af69-032d293f26af" containerName="istio-proxy"
Jul 10 23:40:44.293971 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.293928    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="906d9f75-9dfe-4014-b699-22e47c5c0c40" containerName="istio-proxy"
Jul 10 23:40:44.293971 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.293934    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="906d9f75-9dfe-4014-b699-22e47c5c0c40" containerName="istio-proxy"
Jul 10 23:40:44.293971 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.293943    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b09f957d-74a9-494e-9fe7-e29c9aa15baf" containerName="main"
Jul 10 23:40:44.293971 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.293962    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b09f957d-74a9-494e-9fe7-e29c9aa15baf" containerName="main"
Jul 10 23:40:44.293971 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.293973    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10c229cb-0d97-4fc0-af69-032d293f26af" containerName="istio-proxy"
Jul 10 23:40:44.294208 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.293979    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="10c229cb-0d97-4fc0-af69-032d293f26af" containerName="istio-proxy"
Jul 10 23:40:44.294208 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.293989    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e7ab61b-0379-445c-8080-538f710aedaf" containerName="maas-api"
Jul 10 23:40:44.294208 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.293995    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e7ab61b-0379-445c-8080-538f710aedaf" containerName="maas-api"
Jul 10 23:40:44.294208 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.294060    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e7ab61b-0379-445c-8080-538f710aedaf" containerName="maas-api"
Jul 10 23:40:44.294208 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.294069    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="10c229cb-0d97-4fc0-af69-032d293f26af" containerName="istio-proxy"
Jul 10 23:40:44.294208 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.294075    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="10c229cb-0d97-4fc0-af69-032d293f26af" containerName="istio-proxy"
Jul 10 23:40:44.294208 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.294081    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="906d9f75-9dfe-4014-b699-22e47c5c0c40" containerName="istio-proxy"
Jul 10 23:40:44.294208 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.294087    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="906d9f75-9dfe-4014-b699-22e47c5c0c40" containerName="istio-proxy"
Jul 10 23:40:44.294208 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.294092    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b09f957d-74a9-494e-9fe7-e29c9aa15baf" containerName="main"
Jul 10 23:40:44.294208 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.294143    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="906d9f75-9dfe-4014-b699-22e47c5c0c40" containerName="istio-proxy"
Jul 10 23:40:44.294208 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.294148    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="906d9f75-9dfe-4014-b699-22e47c5c0c40" containerName="istio-proxy"
Jul 10 23:40:44.298821 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.298797    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pjdll/must-gather-lql29"
Jul 10 23:40:44.301136 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.301112    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pjdll\"/\"kube-root-ca.crt\""
Jul 10 23:40:44.301265 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.301116    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pjdll\"/\"default-dockercfg-8j987\""
Jul 10 23:40:44.301971 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.301951    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pjdll\"/\"openshift-service-ca.crt\""
Jul 10 23:40:44.308098 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.308069    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pjdll/must-gather-lql29"]
Jul 10 23:40:44.309713 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.309677    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:40:44.366296 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.366257    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8rzg\" (UniqueName: \"kubernetes.io/projected/97c53590-05ac-44ee-b9ed-d3acbb1b8967-kube-api-access-n8rzg\") pod \"must-gather-lql29\" (UID: \"97c53590-05ac-44ee-b9ed-d3acbb1b8967\") " pod="openshift-must-gather-pjdll/must-gather-lql29"
Jul 10 23:40:44.366506 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.366320    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/97c53590-05ac-44ee-b9ed-d3acbb1b8967-must-gather-output\") pod \"must-gather-lql29\" (UID: \"97c53590-05ac-44ee-b9ed-d3acbb1b8967\") " pod="openshift-must-gather-pjdll/must-gather-lql29"
Jul 10 23:40:44.373586 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.373561    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log" limit=1024
Jul 10 23:40:44.467843 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.467802    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8rzg\" (UniqueName: \"kubernetes.io/projected/97c53590-05ac-44ee-b9ed-d3acbb1b8967-kube-api-access-n8rzg\") pod \"must-gather-lql29\" (UID: \"97c53590-05ac-44ee-b9ed-d3acbb1b8967\") " pod="openshift-must-gather-pjdll/must-gather-lql29"
Jul 10 23:40:44.468009 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.467859    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/97c53590-05ac-44ee-b9ed-d3acbb1b8967-must-gather-output\") pod \"must-gather-lql29\" (UID: \"97c53590-05ac-44ee-b9ed-d3acbb1b8967\") " pod="openshift-must-gather-pjdll/must-gather-lql29"
Jul 10 23:40:44.468196 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.468181    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/97c53590-05ac-44ee-b9ed-d3acbb1b8967-must-gather-output\") pod \"must-gather-lql29\" (UID: \"97c53590-05ac-44ee-b9ed-d3acbb1b8967\") " pod="openshift-must-gather-pjdll/must-gather-lql29"
Jul 10 23:40:44.476148 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.476112    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8rzg\" (UniqueName: \"kubernetes.io/projected/97c53590-05ac-44ee-b9ed-d3acbb1b8967-kube-api-access-n8rzg\") pod \"must-gather-lql29\" (UID: \"97c53590-05ac-44ee-b9ed-d3acbb1b8967\") " pod="openshift-must-gather-pjdll/must-gather-lql29"
Jul 10 23:40:44.608842 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.608750    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pjdll/must-gather-lql29"
Jul 10 23:40:44.747464 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.747423    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pjdll/must-gather-lql29"]
Jul 10 23:40:44.749145 ip-10-0-141-239 kubenswrapper[2563]: W0710 23:40:44.749114    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod97c53590_05ac_44ee_b9ed_d3acbb1b8967.slice/crio-36ae21e961bb11319d2ad28be8b006cd0697b794870cb0a7f7a679bf98adae7b WatchSource:0}: Error finding container 36ae21e961bb11319d2ad28be8b006cd0697b794870cb0a7f7a679bf98adae7b: Status 404 returned error can't find the container with id 36ae21e961bb11319d2ad28be8b006cd0697b794870cb0a7f7a679bf98adae7b
Jul 10 23:40:44.751520 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.751502    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 23:40:44.914424 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:44.914321    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pjdll/must-gather-lql29" event={"ID":"97c53590-05ac-44ee-b9ed-d3acbb1b8967","Type":"ContainerStarted","Data":"36ae21e961bb11319d2ad28be8b006cd0697b794870cb0a7f7a679bf98adae7b"}
Jul 10 23:40:45.936111 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:45.936008    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pjdll/must-gather-lql29" event={"ID":"97c53590-05ac-44ee-b9ed-d3acbb1b8967","Type":"ContainerStarted","Data":"a75264125271dbc1ca6be57efbdd4a02b55460ff5f4fc8c5b0743a9e91ef146b"}
Jul 10 23:40:45.936580 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:45.936130    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pjdll/must-gather-lql29" event={"ID":"97c53590-05ac-44ee-b9ed-d3acbb1b8967","Type":"ContainerStarted","Data":"ebd960b88e1279469d627b6883c040fca964a886b11230c870f4e651abf1e320"}
Jul 10 23:40:45.953440 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:45.953355    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pjdll/must-gather-lql29" podStartSLOduration=1.12616044 podStartE2EDuration="1.953331895s" totalImagesPullingTime="827.171455ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 23:40:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:40:45.950142654 +0000 UTC m=+2706.836878795" watchObservedRunningTime="2026-07-10 23:40:45.953331895 +0000 UTC m=+2706.840068039"
Jul 10 23:40:47.116946 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:47.116915    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7zb67_3734910e-9249-4060-afc4-55140bcb9911/global-pull-secret-syncer/0.log"
Jul 10 23:40:47.247628 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:47.247591    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-fwk45_0ee6c937-5bad-4104-916a-71224add52b5/konnectivity-agent/0.log"
Jul 10 23:40:47.390927 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:47.390832    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-239.ec2.internal_6c99c1f4b605d867ddb0d2d31d3a10b5/haproxy/0.log"
Jul 10 23:40:52.517320 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:52.517286    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-684dfc7c65-gssr4_78f28a98-48be-4320-ace0-1b5297b6fbe0/authorino/0.log"
Jul 10 23:40:52.939391 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:52.939208    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-xr96m_09245303-0104-4b38-86ed-caa866cbf92d/manager/0.log"
Jul 10 23:40:54.386826 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:54.386786    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b9051d20-f72c-4818-9624-3c96503459a5/alertmanager/0.log"
Jul 10 23:40:54.435896 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:54.435870    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b9051d20-f72c-4818-9624-3c96503459a5/config-reloader/0.log"
Jul 10 23:40:54.462828 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:54.462740    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b9051d20-f72c-4818-9624-3c96503459a5/kube-rbac-proxy-web/0.log"
Jul 10 23:40:54.483897 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:54.483872    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b9051d20-f72c-4818-9624-3c96503459a5/kube-rbac-proxy/0.log"
Jul 10 23:40:54.503521 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:54.503495    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b9051d20-f72c-4818-9624-3c96503459a5/kube-rbac-proxy-metric/0.log"
Jul 10 23:40:54.523638 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:54.523561    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b9051d20-f72c-4818-9624-3c96503459a5/prom-label-proxy/0.log"
Jul 10 23:40:54.547108 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:54.547082    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b9051d20-f72c-4818-9624-3c96503459a5/init-config-reloader/0.log"
Jul 10 23:40:54.601852 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:54.601819    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-9tbg6_dae10db7-61ca-4e1e-80f2-ab4fb81ebde2/cluster-monitoring-operator/0.log"
Jul 10 23:40:54.920549 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:54.920512    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q8r92_52440bd8-c787-4a9b-99e9-797246119711/node-exporter/0.log"
Jul 10 23:40:54.944856 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:54.944830    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q8r92_52440bd8-c787-4a9b-99e9-797246119711/kube-rbac-proxy/0.log"
Jul 10 23:40:54.976648 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:54.976582    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q8r92_52440bd8-c787-4a9b-99e9-797246119711/init-textfile/0.log"
Jul 10 23:40:55.094826 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:55.094780    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_df28b342-974c-4b66-95e1-15457303a2d7/prometheus/0.log"
Jul 10 23:40:55.117380 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:55.117276    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_df28b342-974c-4b66-95e1-15457303a2d7/config-reloader/0.log"
Jul 10 23:40:55.138743 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:55.138709    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_df28b342-974c-4b66-95e1-15457303a2d7/thanos-sidecar/0.log"
Jul 10 23:40:55.161482 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:55.161449    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_df28b342-974c-4b66-95e1-15457303a2d7/kube-rbac-proxy-web/0.log"
Jul 10 23:40:55.181541 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:55.181509    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_df28b342-974c-4b66-95e1-15457303a2d7/kube-rbac-proxy/0.log"
Jul 10 23:40:55.203174 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:55.203106    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_df28b342-974c-4b66-95e1-15457303a2d7/kube-rbac-proxy-thanos/0.log"
Jul 10 23:40:55.223714 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:55.223683    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_df28b342-974c-4b66-95e1-15457303a2d7/init-config-reloader/0.log"
Jul 10 23:40:55.410172 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:55.410144    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-776454d5b4-n8tbg_39f19430-c21f-45ee-aae2-b8fec526f77f/thanos-query/0.log"
Jul 10 23:40:55.431355 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:55.431314    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-776454d5b4-n8tbg_39f19430-c21f-45ee-aae2-b8fec526f77f/kube-rbac-proxy-web/0.log"
Jul 10 23:40:55.455673 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:55.455647    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-776454d5b4-n8tbg_39f19430-c21f-45ee-aae2-b8fec526f77f/kube-rbac-proxy/0.log"
Jul 10 23:40:55.474032 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:55.473995    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-776454d5b4-n8tbg_39f19430-c21f-45ee-aae2-b8fec526f77f/prom-label-proxy/0.log"
Jul 10 23:40:55.498554 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:55.498512    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-776454d5b4-n8tbg_39f19430-c21f-45ee-aae2-b8fec526f77f/kube-rbac-proxy-rules/0.log"
Jul 10 23:40:55.518226 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:55.518177    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-776454d5b4-n8tbg_39f19430-c21f-45ee-aae2-b8fec526f77f/kube-rbac-proxy-metrics/0.log"
Jul 10 23:40:56.894402 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:56.894343    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-l9t9h_2981ee03-5541-484c-8226-2d606103d31e/networking-console-plugin/0.log"
Jul 10 23:40:58.099495 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:58.099456    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-945ss_24c76975-a76f-48e0-b1a8-098242d58511/download-server/0.log"
Jul 10 23:40:58.264762 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:58.264725    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pjdll/perf-node-gather-daemonset-6lbsc"]
Jul 10 23:40:58.273238 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:58.273205    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pjdll/perf-node-gather-daemonset-6lbsc"
Jul 10 23:40:58.278722 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:58.278685    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pjdll/perf-node-gather-daemonset-6lbsc"]
Jul 10 23:40:58.421426 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:58.421299    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4f76092e-fb10-44e6-9e04-da611c956a20-lib-modules\") pod \"perf-node-gather-daemonset-6lbsc\" (UID: \"4f76092e-fb10-44e6-9e04-da611c956a20\") " pod="openshift-must-gather-pjdll/perf-node-gather-daemonset-6lbsc"
Jul 10 23:40:58.421426 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:58.421353    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4f76092e-fb10-44e6-9e04-da611c956a20-proc\") pod \"perf-node-gather-daemonset-6lbsc\" (UID: \"4f76092e-fb10-44e6-9e04-da611c956a20\") " pod="openshift-must-gather-pjdll/perf-node-gather-daemonset-6lbsc"
Jul 10 23:40:58.421632 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:58.421504    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4f76092e-fb10-44e6-9e04-da611c956a20-podres\") pod \"perf-node-gather-daemonset-6lbsc\" (UID: \"4f76092e-fb10-44e6-9e04-da611c956a20\") " pod="openshift-must-gather-pjdll/perf-node-gather-daemonset-6lbsc"
Jul 10 23:40:58.421632 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:58.421546    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f76092e-fb10-44e6-9e04-da611c956a20-sys\") pod \"perf-node-gather-daemonset-6lbsc\" (UID: \"4f76092e-fb10-44e6-9e04-da611c956a20\") " pod="openshift-must-gather-pjdll/perf-node-gather-daemonset-6lbsc"
Jul 10 23:40:58.421632 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:58.421573    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrjnz\" (UniqueName: \"kubernetes.io/projected/4f76092e-fb10-44e6-9e04-da611c956a20-kube-api-access-qrjnz\") pod \"perf-node-gather-daemonset-6lbsc\" (UID: \"4f76092e-fb10-44e6-9e04-da611c956a20\") " pod="openshift-must-gather-pjdll/perf-node-gather-daemonset-6lbsc"
Jul 10 23:40:58.523423 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:58.522536    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4f76092e-fb10-44e6-9e04-da611c956a20-lib-modules\") pod \"perf-node-gather-daemonset-6lbsc\" (UID: \"4f76092e-fb10-44e6-9e04-da611c956a20\") " pod="openshift-must-gather-pjdll/perf-node-gather-daemonset-6lbsc"
Jul 10 23:40:58.523423 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:58.522597    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4f76092e-fb10-44e6-9e04-da611c956a20-proc\") pod \"perf-node-gather-daemonset-6lbsc\" (UID: \"4f76092e-fb10-44e6-9e04-da611c956a20\") " pod="openshift-must-gather-pjdll/perf-node-gather-daemonset-6lbsc"
Jul 10 23:40:58.523423 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:58.522669    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4f76092e-fb10-44e6-9e04-da611c956a20-podres\") pod \"perf-node-gather-daemonset-6lbsc\" (UID: \"4f76092e-fb10-44e6-9e04-da611c956a20\") " pod="openshift-must-gather-pjdll/perf-node-gather-daemonset-6lbsc"
Jul 10 23:40:58.523423 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:58.522691    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f76092e-fb10-44e6-9e04-da611c956a20-sys\") pod \"perf-node-gather-daemonset-6lbsc\" (UID: \"4f76092e-fb10-44e6-9e04-da611c956a20\") " pod="openshift-must-gather-pjdll/perf-node-gather-daemonset-6lbsc"
Jul 10 23:40:58.523423 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:58.522714    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrjnz\" (UniqueName: \"kubernetes.io/projected/4f76092e-fb10-44e6-9e04-da611c956a20-kube-api-access-qrjnz\") pod \"perf-node-gather-daemonset-6lbsc\" (UID: \"4f76092e-fb10-44e6-9e04-da611c956a20\") " pod="openshift-must-gather-pjdll/perf-node-gather-daemonset-6lbsc"
Jul 10 23:40:58.523423 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:58.523187    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4f76092e-fb10-44e6-9e04-da611c956a20-lib-modules\") pod \"perf-node-gather-daemonset-6lbsc\" (UID: \"4f76092e-fb10-44e6-9e04-da611c956a20\") " pod="openshift-must-gather-pjdll/perf-node-gather-daemonset-6lbsc"
Jul 10 23:40:58.523423 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:58.523240    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4f76092e-fb10-44e6-9e04-da611c956a20-proc\") pod \"perf-node-gather-daemonset-6lbsc\" (UID: \"4f76092e-fb10-44e6-9e04-da611c956a20\") " pod="openshift-must-gather-pjdll/perf-node-gather-daemonset-6lbsc"
Jul 10 23:40:58.523423 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:58.523329    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4f76092e-fb10-44e6-9e04-da611c956a20-podres\") pod \"perf-node-gather-daemonset-6lbsc\" (UID: \"4f76092e-fb10-44e6-9e04-da611c956a20\") " pod="openshift-must-gather-pjdll/perf-node-gather-daemonset-6lbsc"
Jul 10 23:40:58.523423 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:58.523387    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f76092e-fb10-44e6-9e04-da611c956a20-sys\") pod \"perf-node-gather-daemonset-6lbsc\" (UID: \"4f76092e-fb10-44e6-9e04-da611c956a20\") " pod="openshift-must-gather-pjdll/perf-node-gather-daemonset-6lbsc"
Jul 10 23:40:58.530827 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:58.530798    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrjnz\" (UniqueName: \"kubernetes.io/projected/4f76092e-fb10-44e6-9e04-da611c956a20-kube-api-access-qrjnz\") pod \"perf-node-gather-daemonset-6lbsc\" (UID: \"4f76092e-fb10-44e6-9e04-da611c956a20\") " pod="openshift-must-gather-pjdll/perf-node-gather-daemonset-6lbsc"
Jul 10 23:40:58.588420 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:58.588377    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pjdll/perf-node-gather-daemonset-6lbsc"
Jul 10 23:40:58.783989 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:58.783805    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pjdll/perf-node-gather-daemonset-6lbsc"]
Jul 10 23:40:58.835740 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:58.835712    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-24jvw_76f2a250-6c88-458c-91dc-b89e97416adc/volume-data-source-validator/0.log"
Jul 10 23:40:59.035470 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:59.035428    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pjdll/perf-node-gather-daemonset-6lbsc" event={"ID":"4f76092e-fb10-44e6-9e04-da611c956a20","Type":"ContainerStarted","Data":"f5f6b78d05f6424b50104114a0b9e0fe00feb52e6b50b054fe3c2245f54ce79e"}
Jul 10 23:40:59.035657 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:59.035498    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pjdll/perf-node-gather-daemonset-6lbsc"
Jul 10 23:40:59.035657 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:59.035524    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pjdll/perf-node-gather-daemonset-6lbsc" event={"ID":"4f76092e-fb10-44e6-9e04-da611c956a20","Type":"ContainerStarted","Data":"02dc55f7f472bef69cc2dbbee3116d4efb6dd9ffc21a27aa2e2184dcc0d38ae6"}
Jul 10 23:40:59.051620 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:59.051552    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pjdll/perf-node-gather-daemonset-6lbsc" podStartSLOduration=1.051529133 podStartE2EDuration="1.051529133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 23:40:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 23:40:59.049526356 +0000 UTC m=+2719.936262500" watchObservedRunningTime="2026-07-10 23:40:59.051529133 +0000 UTC m=+2719.938265278"
Jul 10 23:40:59.929354 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:59.929313    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v2p2h_741d147a-735b-40ef-a55b-8e8f03b062d0/dns/0.log"
Jul 10 23:40:59.947488 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:40:59.947460    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v2p2h_741d147a-735b-40ef-a55b-8e8f03b062d0/kube-rbac-proxy/0.log"
Jul 10 23:41:00.012321 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:00.012283    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kd62p_54d84e33-5ba4-4b6c-8c8f-4ddaac24eb03/dns-node-resolver/0.log"
Jul 10 23:41:00.537441 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:00.537408    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-298xj_e17cacf8-0106-484f-a226-2f584a8fc5a5/node-ca/0.log"
Jul 10 23:41:01.738248 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:01.738221    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7947c57c45-c46r7_98989cf4-283f-4a9d-82b0-9b4f967d39c9/kube-auth-proxy/0.log"
Jul 10 23:41:02.605571 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:02.605514    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9ttw7_67dc0695-fad8-4ecf-bc13-3a7410c1b195/serve-healthcheck-canary/0.log"
Jul 10 23:41:03.111228 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:03.111170    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-hwd7d_24725b9a-723c-41d5-959d-4ae184f4d456/insights-operator/0.log"
Jul 10 23:41:03.111807 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:03.111540    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-hwd7d_24725b9a-723c-41d5-959d-4ae184f4d456/insights-operator/1.log"
Jul 10 23:41:03.130891 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:03.130859    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8c7sr_67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d/kube-rbac-proxy/0.log"
Jul 10 23:41:03.149764 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:03.149726    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8c7sr_67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d/exporter/0.log"
Jul 10 23:41:03.169040 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:03.169011    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8c7sr_67b7ba69-2f4d-49f9-a2e1-893d5ebfee7d/extractor/0.log"
Jul 10 23:41:05.056106 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:05.056071    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pjdll/perf-node-gather-daemonset-6lbsc"
Jul 10 23:41:05.739022 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:05.738980    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58bf55498c-lxs9p_96854ace-3322-4255-94bd-eb446df0a254/manager/0.log"
Jul 10 23:41:05.759221 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:05.759191    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58bf55498c-lxs9p_96854ace-3322-4255-94bd-eb446df0a254/copy-manifests/0.log"
Jul 10 23:41:07.084265 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:07.084232    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-85bcb67fdb-224bc_ad95df28-76f7-4c62-ac24-858d7303eaed/manager/0.log"
Jul 10 23:41:07.132308 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:07.132269    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-2n95f_65f11176-0b83-48c0-ac77-fd3bd43ac0df/openshift-lws-operator/0.log"
Jul 10 23:41:08.921691 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:08.921613    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7947c57c45-c46r7_98989cf4-283f-4a9d-82b0-9b4f967d39c9/kube-auth-proxy/0.log"
Jul 10 23:41:10.557921 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:10.557890    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7947c57c45-c46r7_98989cf4-283f-4a9d-82b0-9b4f967d39c9/kube-auth-proxy/0.log"
Jul 10 23:41:12.475620 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:12.475586    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-kgdvg_79faa6c0-9a5c-47b4-b78c-c81181096ba6/migrator/0.log"
Jul 10 23:41:12.498181 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:12.498152    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-kgdvg_79faa6c0-9a5c-47b4-b78c-c81181096ba6/graceful-termination/0.log"
Jul 10 23:41:12.936423 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:12.936386    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-bxm84_5dd8a036-d481-4d56-9e10-1dc8e52983ac/kube-storage-version-migrator-operator/1.log"
Jul 10 23:41:12.943228 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:12.938059    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-bxm84_5dd8a036-d481-4d56-9e10-1dc8e52983ac/kube-storage-version-migrator-operator/0.log"
Jul 10 23:41:14.204522 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:14.204491    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xpkzr_cdbc917d-2667-438e-928b-326933a924d8/kube-multus-additional-cni-plugins/0.log"
Jul 10 23:41:14.223352 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:14.223320    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xpkzr_cdbc917d-2667-438e-928b-326933a924d8/egress-router-binary-copy/0.log"
Jul 10 23:41:14.243185 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:14.243156    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xpkzr_cdbc917d-2667-438e-928b-326933a924d8/cni-plugins/0.log"
Jul 10 23:41:14.261830 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:14.261801    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xpkzr_cdbc917d-2667-438e-928b-326933a924d8/bond-cni-plugin/0.log"
Jul 10 23:41:14.281856 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:14.281818    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xpkzr_cdbc917d-2667-438e-928b-326933a924d8/routeoverride-cni/0.log"
Jul 10 23:41:14.300322 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:14.300294    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xpkzr_cdbc917d-2667-438e-928b-326933a924d8/whereabouts-cni-bincopy/0.log"
Jul 10 23:41:14.319356 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:14.319283    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xpkzr_cdbc917d-2667-438e-928b-326933a924d8/whereabouts-cni/0.log"
Jul 10 23:41:14.387167 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:14.387138    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-rdxmp_b8382cd0-6bcd-4037-814a-dfa9c652d3bd/kube-multus/0.log"
Jul 10 23:41:14.447855 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:14.447825    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-w9kxw_9829a27f-3dbd-44c7-b278-f26beb69f82a/network-metrics-daemon/0.log"
Jul 10 23:41:14.467493 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:14.467403    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-w9kxw_9829a27f-3dbd-44c7-b278-f26beb69f82a/kube-rbac-proxy/0.log"
Jul 10 23:41:15.807446 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:15.807417    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/ovn-controller/0.log"
Jul 10 23:41:15.823703 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:15.823647    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/ovn-acl-logging/0.log"
Jul 10 23:41:15.823918 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:15.823895    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/ovn-acl-logging/1.log"
Jul 10 23:41:15.844896 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:15.844865    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/kube-rbac-proxy-node/0.log"
Jul 10 23:41:15.866599 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:15.866527    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/kube-rbac-proxy-ovn-metrics/0.log"
Jul 10 23:41:15.883473 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:15.883439    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/northd/0.log"
Jul 10 23:41:15.905925 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:15.905898    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/nbdb/0.log"
Jul 10 23:41:15.924953 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:15.924921    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/sbdb/0.log"
Jul 10 23:41:15.990458 ip-10-0-141-239 kubenswrapper[2563]: I0710 23:41:15.990420    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qj86t_861ff3e2-b6a3-4a8b-a23f-88faf0b7e203/ovnkube-controller/0.log"